Ground Control

TLDR
Ground Control is a unified data analytics platform for over 20 labeling tools used by 6,000+ annotators. Built to streamline data handling, improve efficiency, and provide real-time analytics for both enterprise customers and annotators, Ground Control delivered significant operational efficiencies, drove revenue growth, and ensured compliance with industry standards. The platform resolved 2,500+ dataset edge cases for major clients like Cruise and John Deere. Its scalability and real-time insights helped secure new contracts and renewals, contributing over $3 million in revenue.

As the Senior Product Manager for iMerit’s Ground Control, I was responsible for defining and continuously refining the product’s vision and purpose based on feedback from internal and external stakeholders- the What and the Why. My role included identifying and prioritizing the most impactful features, ensuring platform compliance, and collaborating closely with Engineering and Program Management to establish the implementation timeline and technical approach- the How and the When.

Problem Statement

An annotation workforce of 6000+ annotators use 20+ different labeling tools, across 5 time zones, to complete data labeling tasks for 60+ enterprise customers all with different labeling requirements.

How do you consolidate & parse diverse data from multiple sources to create an accurate real-time macro and micro view of the annotation workforce's performance, enabling: proactive issue triage, edge case resolution, best practice identification and implementation, and the discovery of new products or services for the data enrichment marketplace?

Initiation & High Level Requirements

Requirements include:

  1. Unified Platform: Consolidate multiple data types and analytics tools into a single-source-of-truth platform.
  2. Scalability: Support the large-scale operations of over 6000 annotators and petabytes of data .
  3. Data Integration: Seamlessly integrate data in various formats (e.g., images, videos, text, etc.) from various sources, including internal iMerit tools, external enterprise tools (Labelbox, Appen, Dataloop, etc.), and client tools. (Note: this is more challenging than appears, due to tools being in various states of maturity and necessary data not often easy to access/extract.)
  4. Real-time Analytics: Provide ability for near-real-time/real-time insights and analytics for internal users and external clients.
  5. Anomaly Detection: Implement easy-to-use, easy-to-scale mechanisms for detecting and resolving dataset anomalies.
  6. Security & Compliance: Ensure compliance with industry standards (ISO27001, HIPAA, GDPR, SOC2). Provide enterprise-grade security and data governance.
  7. User Management: Include features for workforce management and skill mapping.

Development & Initiation

The development of Ground Control began with a clear vision to create an end-to-end business intelligence solution tailored for enterprise AI. The platform was designed to:

  1. Integrate Various Data Sources: Allowing input from browser plugins, APIs, manual inputs, and embedded plugins.
  2. Enable Real-time Monitoring: Through customizable dynamic dashboards and self-serve analytics.
  3. Support Advanced Analytics: Including predictive analytics, machine learning services, and business insights.
  4. Ensure Security: Implementing robust data governance frameworks, role-based access controls, and encryption.
  5. Automate Processes: Introducing automation in data annotation, anomaly management, and workflow configuration.

Results & Impact

  1. Operational Efficiency: Improved annotation workforce efficiency, significantly increasing throughput and tracking capabilities- hugely relevant for proactive BQM (bottom-quartile-management), before low performers impacted total contract progress.
  2. Revenue Growth: Supported enterprise customers, providing analytics and insights that directly led to securing new or expanding existing contracts with top enterprise customers like John Deere and Cruise.
  3. Real-time Insights: Delivered on-demand, customized business insights, improving decision-making and operational efficiency, and ingest data from any tool- hugely important to clients with tools that were either broken or still in development.
  4. Scalability: Scaled data platform operations by 50% to accommodate a growing number of projects and customers.
  5. Compliance and Security: Ensured compliance with ISO27001, HIPAA, GDPR, and SOC2 standards, guaranteeing secure and reliable data handling.
  6. Anomaly Detection & Resolution: Captured and resolved thousands of dataset anomalies (edge cases) across various labeling tools, providing customers with critical insights for model improvement. For example, using the Edge Case Module (diagram below), iMerit was able to capture and resolve 2500+ unique data set anomalies for Cruise. These findings enabled additional subject expert training.
  7. Immediate & Scalable Data Ingestion: Developed the Cloud Cover Chrome extension, enabling immediate real-time user activity tracking and enhancing customer trust and workflow optimization.

"Ground Control convinced us to not renew our contract with our other labeling vendors and give 100% of our annotation work to iMerit." -customer quote from an Israeli AI startup

Lessons Learned

  1. Solve Core Problems: Addressing 80% of an enterprise customer's 'unique' problem often can uncover similar problems with other enterprise customers, which facilitates a more flexible platform design and focuses the team on working on the right thing.
  2. Proactively Problem Solve: Early identification and proactive resolution of potential issues prevent project delays, increase client trust, and maintain workflow continuity.
  3. Team Culture > everything: Team culture is everything. Agree on standards on how to work, then hold everyone accountable to that agreed upon quality bar.
  4. Scalable Architecture: Design with scalability in mind from the beginning. This design-centric approach helps in managing growth and accommodating future expansions without major overhauls.
  5. User-Centric Design: Prioritizing user experience and feedback is crucial for adoption and usability. Iterative design and regular user testing lead to more effective and user-friendly tools. Dog food your product! Also, if you're in a margins-business, focus on what the client cares about: a cheaper, faster, higher quality service!

Bottom Line

Ground Control has proven to be a pivotal solution for iMerit, providing a comprehensive Business Intelligence & analytics platform that meets the diverse needs of its large-scale annotation workforce and enterprise customers. The integration of advanced analytics, real-time insights, and robust data governance has significantly improved operational efficiency, unlocked additional revenue, and helps establish iMerit as a leader in data labeling and business intelligence solution.