The first dataset quality system
backed by real training data.

Three innovations no other quality system has

LQS v3.0 is not a checklist. It's a quality intelligence engine that predicts how well your model will perform if you train on this data — conditioned on your specific task, stress-tested for robustness, and backed by three independent sources of evidence.

The trust mark is awarded only when all three evidence sources are available, confidence level is HIGH, and the composite score is ≥ 60. Like UL Listed for electronics or USDA Organic for food — it means the dataset passed a verification pipeline that predicts downstream performance, not just structural checks.

Calibration study: per-dimension correlation with model performance

We trained 3-model ensemble baselines (XGBoost + Random Forest + Logistic Regression for tabular; TF-IDF LogReg for NLP) on 49 anchor datasets across 7 task types, measured held-out macro F1 via 5-fold cross-validation, and computed Spearman rank correlation between each LQS dimension and downstream performance. Three dimensions are statistically significant (p<0.05). The weights update automatically as more datasets flow through the system.

Per-dimension Spearman ρ vs. downstream macro F1

Calibration suite: 49 anchor datasets across 7 task types (multiclass classification, binary classification, text classification, legal AI, medical AI, financial AI, regression). Baseline models: 3-model ensemble (XGBoost + Random Forest + Logistic Regression) for tabular; TF-IDF Logistic Regression for NLP (DistilBERT attempted first, falls back on timeout). 5-fold stratified cross-validation, fixed seed for reproducibility. The system recalibrates daily via a dual-signal feedback loop: buyer-reported training outcomes (gold standard) and validation benchmark F1 from every dataset processed through the marketplace. Raw data published at calibration/results/.

How fragile is the data under noise?

We systematically flip 1%, 5%, 10%, and 20% of labels to random incorrect values, retrain the baseline model each time, and measure how much F1 degrades. The resulting degradation curve tells you whether the dataset is robust enough for production — before you spend compute discovering it isn't.

Robustness tiers: Robust (≥85) — degrades gracefully, production-ready. Moderate (65–84) — some noise sensitivity, label auditing recommended. Fragile (<65) — high sensitivity, requires quality control before training. Tested across 49 datasets with 3-model ensemble baselines and 4 noise injection rates.

Same dataset, different score — because tasks have different needs

A single quality number is misleading. A dataset with perfect class balance is critical for fraud detection but irrelevant for NER. LQS v3.0 reweights the 14 dimensions based on your intended task, producing a score that predicts performance for YOUR specific use case.

How it works: Profiles marked with dataset counts are empirically derived — the multipliers come from per-task Spearman correlations computed against the calibration suite. "Expert-tuned" profiles use domain-knowledge multipliers and will be upgraded to empirical once enough datasets of that type flow through the marketplace. All multipliers are renormalized to sum to 1.0. Task profiles auto-detect from dataset category and format. The system recalibrates task multipliers daily as new datasets are validated.

Three sources must agree before we call it "verified"

A quality score backed by one methodology is a claim. A score backed by three independent methodologies that agree is evidence. LQS v3.0 requires convergence across statistical analysis, real model training, and adversarial stress testing before awarding HIGH confidence.

Composite score

The v3.0 composite blends all three evidence sources. Statistical scoring provides the base. Empirical training adjusts ±30%. Adversarial robustness adjusts ±15%. The final number is a prediction of downstream performance, not a checklist result.

12 format types

LQS v3.0 has format-specific analysis pipelines for every major ML data format. Each pipeline extracts the signals most meaningful for that format type.

Version history

Each dataset record stores the LQS version used to compute its scores. Datasets scored under earlier versions retain their original scores and are re-scored to v3.0 on next re-validation.