Track Awesome Seml Updates Weekly

A curated list of articles that cover the software engineering best practices for building machine learning applications.

🏠 Home · 🔍 Search · 🔥 Feed · 📮 Subscribe · ❤️ Sponsor · 😺 SE-ML/awesome-seml · ⭐ 1K · 🏷️ Computer Science

[ Daily / Weekly / Overview ]

Feb 07 - Feb 13, 2022

Tooling

• Aim - Aim is an open source experiment tracking tool.

Dec 06 - Dec 12, 2021

Governance

• Understanding Software-2.0 🎓

Oct 18 - Oct 24, 2021

Tooling

• REVISE: REvealing VIsual biaSEs (⭐91) - Automatically detect bias in visual data sets.

Oct 04 - Oct 10, 2021

Governance

• Beyond Debiasing

May 03 - May 09, 2021

Tooling

• Alibi Detect (⭐1.5k) - Python library focused on outlier, adversarial and drift detection.

• PyTorch Lightning (⭐20k) - The lightweight PyTorch wrapper for high-performance AI research. Scale your models, not the boilerplate.

• Robustness Metrics (⭐418) - Lightweight modules to evaluate the robustness of classification models.

• Seldon Core (⭐3.4k) - An MLOps framework to package, deploy, monitor and manage thousands of production machine learning models on Kubernetes.

• Tensorflow Data Validation (TFDV) (⭐674) - Library for exploring and validating machine learning data. Similar to Great Expectations, but for Tensorflow data.

Apr 26 - May 02, 2021

Governance

• A Human-Centered Interpretability Framework Based on Weight of Evidence 🎓

Mar 22 - Mar 28, 2021

Model Training

• Fairness On The Ground: Applying Algorithmic FairnessApproaches To Production Systems🎓

Mar 15 - Mar 21, 2021

Deployment and Operation

• Building Continuous Integration Services for Machine Learning 🎓

Jan 04 - Jan 10, 2021

Governance

• Inherent trade-offs in the fair determination of risk scores 🎓

Nov 23 - Nov 29, 2020

Deployment and Operation

• Underspecification Presents Challenges for Credibility in Modern Machine Learning 🎓

Governance

• An Architectural Risk Analysis Of Machine Learning Systems

Oct 26 - Nov 01, 2020

Broad Overviews

• AI Engineering: 11 Foundational Practices ⭐

Data Management

• Data Validation for Machine Learning 🎓

Oct 19 - Oct 25, 2020

Tooling

• Archai (⭐373) - Neural architecture search.

• FairLearn - A toolkit to assess and improve the fairness of machine learning models.

• Great Expectations (⭐7.4k) - Data validation and testing with integration in pipelines.

• LiFT (⭐159) - Linkedin fairness toolkit.

• Model Card Toolkit (⭐314) - Streamlines and automates the generation of model cards; for model documentation.

Sep 21 - Sep 27, 2020

Governance

• Closing the AI Accountability Gap: Defining an End-to-End Framework for Internal Algorithmic Auditing 🎓

Aug 03 - Aug 09, 2020

Broad Overviews

• Engineering Best Practices for Machine Learning ⭐

• Hidden Technical Debt in Machine Learning Systems 🎓⭐

• Software Engineering for Machine Learning: A Case Study 🎓⭐

Jun 29 - Jul 05, 2020

Model Training

• Nitpicking Machine Learning Technical Debt

Tooling

• Airflow - Programmatically author, schedule and monitor workflows.

• Data Version Control (DVC) - DVC is a data and ML experiments management tool.

• Facets Overview / Facets Dive - Robust visualizations to aid in understanding machine learning datasets.

• Git Large File System (LFS) - Replaces large files such as datasets with text pointers inside Git.

• HParams (⭐126) - A thoughtful approach to configuration management for machine learning projects.

• Kubeflow - A platform for data scientists who want to build and experiment with ML pipelines.

• Label Studio (⭐11k) - A multi-type data labeling and annotation tool with standardized output format.

• MLFlow - Manage the ML lifecycle, including experimentation, deployment, and a central model registry.

• Neptune.ai - Experiment tracking tool bringing organization and collaboration to data science projects.

• Neuraxle (⭐543) - Sklearn-like framework for hyperparameter tuning and AutoML in deep learning projects.

• OpenML - An inclusive movement to build an open, organized, online ecosystem for machine learning.

• Spark Machine Learning - Spark’s ML library consisting of common learning algorithms and utilities.

• TensorBoard - TensorFlow's Visualization Toolkit.

• Tensorflow Extended (TFX) - An end-to-end platform for deploying production ML pipelines.

• Weights & Biases - Experiment tracking, model optimization, and dataset versioning.

May 18 - May 24, 2020

Governance

• Responsible AI practices ⭐

• Toward Trustworthy AI Development: Mechanisms for Supporting Verifiable Claims

Apr 06 - Apr 12, 2020

Broad Overviews

• Rules of Machine Learning: Best Practices for ML Engineering ⭐

Model Training

• On Comparing Classifiers: Pitfalls to Avoid and a Recommended Approach 🎓⭐

Deployment and Operation

• Continuous Delivery for Machine Learning ⭐

Mar 30 - Apr 05, 2020

Deployment and Operation

• Scaling Machine Learning as a Service🎓

Mar 02 - Mar 08, 2020

Broad Overviews

• Best Practices for Machine Learning Applications

Data Management

• Data management challenges in production machine learning

Model Training

• Apples-to-apples in cross-validation studies: pitfalls in classifier performance measurement 🎓

• Pitfalls and Best Practices in Algorithm Configuration 🎓

• Pitfalls of supervised feature selection 🎓

Feb 24 - Mar 01, 2020

Data Management

• How to organize data labelling for ML

Model Training

• Preparing and Architecting for Machine Learning

Feb 10 - Feb 16, 2020

Data Management

• Automating Large-Scale Data Quality Verification 🎓

• The Data Linter: Lightweight, Automated Sanity Checking for ML Data Sets 🎓

Deployment and Operation

• The ML Test Score: A Rubric for ML Production Readiness and Technical Debt Reduction 🎓

• Versioning for end-to-end machine learning pipelines 🎓

Social Aspects

• Data Scientists in Software Teams: State of the Art and Challenges 🎓

Feb 03 - Feb 09, 2020

Data Management

• A Survey on Data Collection for Machine Learning A Big Data - AI Integration Perspective_2019 🎓

• The curse of big data labeling and three ways to solve it

• The ultimate guide to data labeling for ML

Model Training

• 10 Best Practices for Deep Learning

• How do you manage your Machine Learning Experiments?

• Machine Learning Testing: Survey, Landscapes and Horizons 🎓

• On human intellect and machine failures: Troubleshooting integrative machine learning systems 🎓

• Preliminary Systematic Literature Review of Machine Learning System Development Process 🎓

• Software development best practices in a deep learning environment

• Testing and Debugging in Machine Learning

• What Went Wrong and Why? Diagnosing Situated Interaction Failures in the Wild 🎓

Deployment and Operation

• Best Practices in Machine Learning Infrastructure

• Continuous Training for Production ML in the TensorFlow Extended (TFX) Platform 🎓

• Fairness Indicators: Scalable Infrastructure for Fair ML Systems 🎓

• Machine Learning Logistics

• Machine learning: Moving from experiments to production

• ML Ops: Machine Learning as an engineered disciplined

• Model Governance Reducing the Anarchy of Production 🎓

• ModelOps: Cloud-based lifecycle management for reliable and trusted AI

• Operational Machine Learning

• TFX: A tensorflow-based Production-Scale ML Platform 🎓

Social Aspects

• Machine Learning Interviews (⭐7.4k)

• Managing Machine Learning Projects

• Principled Machine Learning: Practices and Tools for Efficient Collaboration