With the rise of video, Alegion has become the top vendor for video annotation. In a 3 part series covering a year of intense experimentation and development. Today, in the final part of our 3 part series we explore the significance of ground truth video data errors for annotation: how they happen, the most common problems, how competitor platforms handle issues (spoiler: it’s not ideal), and why Alegion chose to go another way.
With the rise of video, Alegion has become the top vendor for video annotation. In a 3 part series covering a year of intense experimentation and development. Today, in part 2 of a 3 part series we reflect on how we built a transformational solution to address the 7 major pain points of video annotation. We knew that if we could solve for the goals of our clients, the benefits would be tremendous across use cases and verticals.
Part 1: Building a Video Annotation Platform by Chip Ray, CTO
Over the past year, Alegion has taken on the challenge of building the best possible video annotation experience. Having assessed the tooling available on the market, we knew there just had to be a better way. At the same time, we were witnessing an explosion of research and focus around video data in the computer vision (CV) space.
Learn about our incident response best practices collected and implemented at Alegion, forged from input provided by industry experts, as well as plenty of hands-on experience managing incidents while growing our platform over the years.
Parrot, one of the leaders in drone systems and technologies, produce a good collection of aerial videos that can be used to train various aerial autonomous vehicles suitable for Engineering, Construction, and Agriculture use cases.
Another good source for Autonomous Vehicle data, The Level 5 Dataset includes over 55,000 human-labeled 3D annotated frames, surface map, and an underlying HD spatial semantic map that are captured from 7 cameras and up to 3 Lidar sensors that can be used to contextualize the data. Note that this dataset uses nuScenes format.
One of the most famous novel Computer Vision benchmark for Autonomous Driving, KITTI dataset contain videos, Velodyne sensors, and a GPS localization system recording of rural areas and highway driving in the city of Karlsruhe. A collaborative work of Karlsruhe Institute of Technology and Toyota Technological Institute at Chicago
This is high-quality geospatial data with precision-labeled and high-resolution satellite imagery. It contains ~27,000 square km of very high-resolution imagery, 811,000 building footprints, and ~20,000 km of road labels to ensure that there is adequate open source data available for geospatial machine learning research.
HACS = Human Action Clips and Segment Dataset is a CV dataset that is good for building models that deals with Recognition and Temporal Localization, using videos of action segments. This dataset contains 1.55M clips on 504K videos.
COCO, short for Common Objects in Context, is large image recognition/classification, object detection, segmentation, and captioning dataset. Volume: 330K images (200K+ annotated); more than 2M instances in 80 object categories, with 5 captions per image, and 250,000 people with key points.
Known as the de-facto image dataset for CV algorithms, ImageNet is a large image database of various quality-controlled and human annotated object images that aims to support Computer Vision researchers and practitioners with the need of more data.
Survey says: model confidence is built on a lot of training data
New survey reveals a concrete definition of “huge” in reference to 72% report that production-level model confidence will require more than 100,000 labeled data items. And 10% need more than 10 million! That’s a lot of bounding boxes and polygons!
What does Alegion do? O'Reilly AI Conference this week. We prepare quality training data for Fortune 1000 companies. We have the people, processes, and platform needed to take training data needs off your plate.
So what is the state of AI expertise? Early stages but growing fast. AI teams are typically lead by a highly trained data scientist. Data scientists are put in charge of all sorts of data-related projects such as data clean up, categorization, structuring, and lots of analysis.
We participated in an AI event during last week’s South by Southwest Conference (SxSW) that perfectly captured the Austin experience. Capital Factory, a local incubator, hosted a Saturday night AI Mixer.
We've integrated computer vision capabilities into our training data platform
In a press release this morning we announced new computer vision capabilities in our Training Data Platform. We've added data labeling and annotation features for image classification, object localization, and semantic segmentation.
The Journey of a Machine Learning (and learning) Project
Alegion partners with clients to create training data for their machine learning projects. We have a good sense of how the enterprise manages ML projects today, and have developed a nomenclature for describing project phases.
7 tips to being AI ready. We produce quality training data for computer vision and natural language processing, which means we have a lot of experience with machine learning projects throughout their lifecycle. Our experience with firsttime ML project teams has given us valuable insights.
There are irreplaceable human beings answering the phone line at 911 who calmly, rationally, and with compassion, help people in need. But there are all sorts of systems and data available, that if connected with one another, could make those invaluable humans response times faster.
Analysts and researchers who have been following the emergence of AI continue to witness it’s growth. More projects, more companies, more departments from those companies, are discovering ways AI can improve efficiencies and customer experiences. It’s still early days, keep your eyes on this space.
2018 saw stories in a huge cross section of publications. Technology standards like ZDNet and TechTarget covered the business of AI. The interest in AI technology breakthroughs spanned many other publications, including Fortune, the Wall St Journal, and Fast Company.
“Human-in-the-loop” is an increasingly popular way of describing the application of human judgement to AI training. And of course, for some types of AI training humans remain an essential ingredient.
"But who are the humans in the loop?" people often ask, and it's a reasonable question.
If the images your algorithm is learning from were shot with a lens with a color filter, all of those images will be identically skewed. If the ruler used to determine the dimensions of data elements was ¼ inch short of a foot, all of the dimensions will be wrong by an identical proportion.
Much like with traditional software development, as AI and ML initiatives take priority on the enterprise agenda, data scientists need to retire the outdated waterfall approach and replace it a more agile style of development.
Prejudicial bias is probably what most people think of when they hear the word. And this kind of bias is the source of many scary AI headlines. And yet, this phenomenon of human prejudice influencing algorithms is complex.
In waterfall these interdependencies aren’t fully explored until the penultimate testing stage, at which point any oversights or miscalculations that are discovered drive the entire project back to the architecture or coding stage.
With sample bias in machine learning in computer vision, natural language processing and entity resolution, it isn’t possible - or even necessary - to expose an algorithm to its entire universe in order to train it. A sample of that universe is adequate, more affordable, and more practical.
ABJ notes that “CEO Nathaniel Gates expects Alegion's headcount to nearly double by the end of the year — and to pivot to becoming ‘product focused’ rather than ‘service focused’ to widen its market reach.