Choosing open-source AI models for data science starts with a license check, because most models described as open source are more accurately open weights. The Open Source Definition requires that “the program must include source code” — and for a model, the analogue is training data and training code, which are almost never released.
On this page
Open source or open weights
The phrase is used loosely and the distinction has legal consequences for a data science team. The Open Source Definition sets ten criteria, and three of them are awkward for model releases:
- Source code: “The program must include source code, and must allow distribution in source code as well as compiled form.” For a trained model, the closest analogue to source is the training data and the code that produced the weights. Published weights are closer to a compiled binary.
- Derived works: “The license must allow modifications and derived works, and must allow them to be distributed under the same terms as the license.”
- No discrimination against fields of endeavor: “The license must not restrict anyone from making use of the program in a specific field of endeavor.”
That last criterion is the one bespoke model licenses most often fail, because acceptable-use policies and usage thresholds are field-of-use restrictions by construction. A release can be genuinely generous, freely downloadable, and commercially usable, and still not be open source under this definition.
None of that makes open-weight models unsuitable. It means the correct description is “open weights,” and it means you have to read the actual license rather than the word on the announcement page. Several model families do ship under standard permissive licenses such as Apache 2.0; several ship under custom community licenses. The difference matters most at the moment your legal team asks.
The license questions to actually ask
Open the license file that accompanies the weights — not the blog post — and answer six questions before you build anything on it.
- Is it a standard license or a bespoke one? A recognized name such as Apache 2.0 or MIT is a known quantity. A custom license needs reading in full.
- Is commercial use permitted without a threshold? Some licenses require a separate agreement above a stated scale.
- Are there field-of-use restrictions? Acceptable-use policies are part of the license, and they bind you.
- What are the attribution and naming obligations? Some require that derivative models carry a specific name prefix, which affects your product naming.
- Can you distribute a fine-tuned derivative, and under what terms?
- Can outputs be used to train other models? Frequently restricted, and frequently overlooked.
Record the answers alongside the model version in your project documentation. Shortlisting open-source AI models for data science without that record is how teams end up unable to ship. Licenses change between model generations from the same vendor, so “we already checked that family” is not a durable answer.
Size the model to your hardware
Capability you cannot load is not capability. Weight memory is parameters times bits per parameter divided by eight, so a 13B model at 4-bit is about 6.5GB and a 70B at 4-bit is about 35GB — which exceeds every consumer graphics card.
Work backwards from the memory you have rather than forwards from a leaderboard. Our guide to running local models on a 16GB card works the arithmetic through in detail, and the same method applies to any card or unified-memory machine.
One counterintuitive rule holds across hardware tiers: a larger model at 4-bit usually beats a smaller model at 8-bit for the same memory. Quantization to 4-bit costs less than halving the parameter count.
Match the model class to the task
| Task | What to look for |
|---|---|
| Code generation and refactoring | A code-specialized model; general models underperform badly here |
| Semantic search over documents | A dedicated embedding model, not a chat model |
| Classification and extraction at volume | The smallest model that passes your accuracy bar — throughput dominates |
| Summarizing long documents | Long context window, and check the model degrades gracefully at length |
| Charts, tables and document images | A vision-language model |
| Reasoning over multi-step problems | A model with an explicit reasoning mode, accepting slower output |
The embedding row is the most common mistake teams make when picking open-source AI models for data science. Teams reach for a large chat model to build a retrieval system when a small purpose-built embedding model is faster, cheaper and more accurate for that job.
Where offline genuinely matters
Running locally is worth engineering effort in four situations: data that cannot leave your infrastructure for contractual or regulatory reasons; workloads where per-token pricing becomes the dominant cost at volume; environments with unreliable connectivity; and research that requires the model to stay fixed for reproducibility, which a hosted endpoint cannot guarantee.
That last point is underrated in data science specifically. A hosted model can change beneath a published result. A pinned local model with a recorded version and quantization is reproducible years later, which for research output is not a convenience but a requirement.
The evaluation checklist
- Read the license file, not the announcement. Record the answers to the six questions above.
- Confirm it fits your memory at a quantization you are willing to accept.
- Pick the model class for the task — embedding, code, vision or general.
- Test on your own data with a fixed task set before committing.
- Pin the exact version and quantization in your project configuration.
- Check the model card for the training-data description, which is what you will be asked about first.
Done in that order, selecting open-source AI models for data science becomes a licensing and memory exercise with a capability check at the end — rather than a leaderboard exercise. Judged this way, the field of open-source AI models for data science narrows fast that produces a model you cannot legally deploy or physically load.
Join the discussion
Add useful context, ask a focused question or share relevant experience. Comments are moderated to protect readers from spam and promotional links.
Leave a thoughtful comment