Fastapi conversion#2071

Draft

bfoley12 wants to merge 175 commits into

devfrom

fastapi-conversion

bfoley12 commented Jun 16, 2026 •

edited

Loading

Collaborator

Summary

Start of major rewrite of the API server to FastAPI and other modern frameworks (pydantic, Beanie, etc.)

This PR is the essential groundwork of the rewrite. However, further PRs are necessary for full functionality.

Major TODOs after this PR

Downloads
- This involves infra design to push data from Contributions and Components into S3 as parquet (maybe delta tables if it goes that far)
Unit normalization (Pint/Unyt)
- Also entails making input data formatted strings, then internally parsing the formats into better representations (objects) before storage
Greatly expand telemetry

Minor TODOs after this PR:

Add indices to Beanie models
Remove notebooks
Remove CIF
Email notifications

bfoley12 self-assigned this

bfoley12 added this to the API Server Rewrite milestone

bfoley12 commented

View reviewed changes

mpcontribs-api/scripts/start.sh

bfoley12 Jun 17, 2026

Collaborator Author

Removing jupyter and local datadog reference

bfoley12 commented

View reviewed changes

mpcontribs-api/scripts/start_rq.sh

bfoley12 Jun 17, 2026

Collaborator Author

Removing jupyter and local datadog reference

bfoley12 commented

View reviewed changes

mpcontribs-api/supervisord/conf.py

bfoley12 Jun 17, 2026

Collaborator Author

Ruff formatting

bfoley12 commented

View reviewed changes

mpcontribs-api/gunicorn.conf.py Outdated

bfoley12 Jun 17, 2026

Collaborator Author

Fastapi uses uvicorn

bfoley12 commented

View reviewed changes

mpcontribs-api/justfile

bfoley12 Jun 17, 2026

Collaborator Author

Convenient command runner

bfoley12 commented

View reviewed changes

mpcontribs-api/Dockerfile

bfoley12 Jun 17, 2026

Collaborator Author

Switch from pip building to uv.
!! Needs further verification

bfoley12 commented

View reviewed changes

mpcontribs-api/main.py

bfoley12 Jun 17, 2026

Collaborator Author

Ruff formatting

bfoley12 commented

View reviewed changes

mpcontribs-api/maintenance.py Outdated

bfoley12 Jun 17, 2026

Collaborator Author

Not using MongoEngine.
!! Still need to implement unit checks, but will be outside of this scope

bfoley12 commented

View reviewed changes

mpcontribs-api/pyproject.toml

bfoley12 Jun 17, 2026

Collaborator Author

Upgrade python to 3.14; move to uv, ruff, and basedpyright; removed outdated/unnecessary packages

tsmathis Jun 17, 2026

Collaborator

the base stack is still at python 3.11

bfoley12 Jun 17, 2026

Collaborator Author

Yeah, I have a note in the Dockerfile about requiring a 3.14 image build - I assumed it would be a good time to bump our python version and use up to date package versions.
Do you think this should be in the scope of the rewrite?

bfoley12 added 19 commits

June 17, 2026 13:38


          Initial ruff formatting and uv project init

94d7227


          Added .venv and .env to gitignore

3c79379


          Moved dev optional-dependencies to a dependency group

4d5c64f


          Synced lock file

8ebb3f6


          Example env vars

98d867f


          Moved to a src/proj style repo. Began creating fastapi server

6558bc7


          Removed project-level uv ennvironment

04da662


          Updated python version to 3.14

b84f81f


          Added projects router and reintroduced prefix

a3624de


          Added Beanie init

f31519b


          Broke redis and kong settings into their own classes for modularity

4de7efd


          Added shared types

48f19a6


          Removed unnecessary files

4e44d03


          Added packages to uv

c44dc0e


          Built out project domain for getting data

da8d2e5


          Added shared dependencies for injection

b130b33


          Added AuthenticationError and GatewayError for more control over thro…

c3004a0

…wn errors


          Ruff formatting and checking

a5b5849


          PrefixedEmail now throws ValidationError instead of ValueError

74ac09f

bfoley12 added 30 commits

June 18, 2026 12:18


          Requests to contributions or their components are now scoped by the u…

e48eb19

…ser - users can no longer operate on components that their contributions dontown


          Enforce contribution-scoped components at service level

2077a49


          Convert id str to objectid

fa292e1


          Readded start_rq so supervisord doesnt fail

7d617ef


          Moved kwargs to new spec

8c162ed


          Changed example kwargs to new specs

8cc2c23


          Added back n start_rq.sh so build succeeds

3acff74


          Added lines to ignore build artifacts

48dce53


          Modernized claude.md


          Pointed setuptools at the right repo - src

23494a1


          Added and fixed tests


          Formatting

0521e7c


          Removing accidental commit of build artifacts

507c11e


          Unified deletion behavior by moving BaseDocumentWithInput to inherit …

afbbee1

…from Document instead of DocumentWithSoftDelet


          Upsert now functions like bulk insert, allowing upserts to fail and r…

19df929

…eporting the successes and failures via BulkWriteSummary


          Improved sensitivity of dict key validation

09c6a25


          Simplified md5 check and improved patch_component_by_id

4a48ef7


          Added a constant for how to handle encoding of polars.DataFrames to h…

cb90bc7

…ave consistent (re)serialization


          Improved dataframe handling to read and write dataframes

1c7a420


          Made ComponentIn to clarify and unify input schema from storage schema

57d906a


          Modified tests to account for new handling


          Added ty for typechecking

408b6a6


          Removed ty - not mature enough

90eaca5


          Changed docstring, imports, and added polars dep

0d122bc


          Commented out unused dependencies

10ce393


          Added custom Access logging in line with previous format.

dbb8c28


          pass exc_info through to logger on upsert error

17e67f7


          Added log_level to errors

dd257b3


          Moved healthcheck router to healthcheck instead of health

0cf8e8f


          Added 'version' to healthcheck for backwards compat

e1096e5

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet