In Defense of Humans - the Fact vs. Fiction of Labeling Automation
The capabilities of ML and AI platforms are constantly evolving, and we spend a lot of time planning, building, and iterating toward a more automated future. However, this forward looking approach can obscure what is fact and what is fiction when it comes to automation and its role in model development.
Better Quantifying The Performance Of Object Detection In Video
Video object detection has several unique challenges. One of the main problems that hinders consistent technical progress is the lack of a comprehensive metric that can evaluate the performance of object detectors in video. We propose the use of a classification aware higher order tracking accuracy (HOTA) metric.
Pixel Tolerance vs. IOU - Which One Should You Use for Quality Training Data?
The purpose of data annotation for computer vision is to teach a model how to identify and classify things. The human mind “annotates” effortlessly - we see something and we identify it (with varying degrees of specificity and accuracy based on prior experience). IOU (intersection over union) and pixel tolerance helps us measure the quality of the annotation.
Responsible AI: How to Mitigate Bias in Your Training Data
The more we use AI to power and automate crucial parts of our daily lives, the more we need to be able to trust that these models are accurate, equitable, and high-performing, which means we need to mitigate bias in our training data.
Alegion Teams with Malaysian Government Agency to Expand ML Workforce Opportunities
Alegion and Malaysian government agency Yayasan Peneraju collaborate to train and certify 1,600 machine learning data labeling professionals, empowering them to earn a sustainable income in the fast-growing AI market.
Calling All Data Scientists - Survey for Alegion 2021 State of the Industry
Each year, Alegion releases our analysis of the AI / ML space. We call upon expert data scientists who are deploying world-changing projects to answer a brief survey on how their projects are going and what challenges they may be running into.
A Step-by-Step Guide to Quality Training Data for Computer Vision
Alegion specializes in high-quality training data for computer vision. In this blog we walk through the 4 prioritized phases to quality training data, and explain the significance of specific metrics for quality
With the rise of video, Alegion has become the top vendor for video annotation. In a 3 part series covering a year of intense experimentation and development. Today, in the final part of our 3 part series we explore the significance of ground truth video data errors for annotation: how they happen, the most common problems, how competitor platforms handle issues (spoiler: it’s not ideal), and why Alegion chose to go another way.
With the rise of video, Alegion has become the top vendor for video annotation. In a 3 part series covering a year of intense experimentation and development. Today, in part 2 of a 3 part series we reflect on how we built a transformational solution to address the 7 major pain points of video annotation. We knew that if we could solve for the goals of our clients, the benefits would be tremendous across use cases and verticals.
Part 1: Building a Video Annotation Platform by Chip Ray, CTO
Over the past year, Alegion has taken on the challenge of building the best possible video annotation experience. Having assessed the tooling available on the market, we knew there just had to be a better way. At the same time, we were witnessing an explosion of research and focus around video data in the computer vision (CV) space.
Learn about our incident response best practices collected and implemented at Alegion, forged from input provided by industry experts, as well as plenty of hands-on experience managing incidents while growing our platform over the years.
Survey says: model confidence is built on a lot of training data
New survey reveals a concrete definition of “huge” in reference to 72% report that production-level model confidence will require more than 100,000 labeled data items. And 10% need more than 10 million! That’s a lot of bounding boxes and polygons!
What does Alegion do? O'Reilly AI Conference this week. We prepare quality training data for Fortune 1000 companies. We have the people, processes, and platform needed to take training data needs off your plate.
So what is the state of AI expertise? Early stages but growing fast. AI teams are typically lead by a highly trained data scientist. Data scientists are put in charge of all sorts of data-related projects such as data clean up, categorization, structuring, and lots of analysis.
We participated in an AI event during last week’s South by Southwest Conference (SxSW) that perfectly captured the Austin experience. Capital Factory, a local incubator, hosted a Saturday night AI Mixer.
We've integrated computer vision capabilities into our training data platform
In a press release this morning we announced new computer vision capabilities in our Training Data Platform. We've added data labeling and annotation features for image classification, object localization, and semantic segmentation.
The Journey of a Machine Learning (and learning) Project
Alegion partners with clients to create training data for their machine learning projects. We have a good sense of how the enterprise manages ML projects today, and have developed a nomenclature for describing project phases.
7 tips to being AI ready. We produce quality training data for computer vision and natural language processing, which means we have a lot of experience with machine learning projects throughout their lifecycle. Our experience with firsttime ML project teams has given us valuable insights.
There are irreplaceable human beings answering the phone line at 911 who calmly, rationally, and with compassion, help people in need. But there are all sorts of systems and data available, that if connected with one another, could make those invaluable humans response times faster.
Analysts and researchers who have been following the emergence of AI continue to witness it’s growth. More projects, more companies, more departments from those companies, are discovering ways AI can improve efficiencies and customer experiences. It’s still early days, keep your eyes on this space.
2018 saw stories in a huge cross section of publications. Technology standards like ZDNet and TechTarget covered the business of AI. The interest in AI technology breakthroughs spanned many other publications, including Fortune, the Wall St Journal, and Fast Company.
“Human-in-the-loop” is an increasingly popular way of describing the application of human judgement to AI training. And of course, for some types of AI training humans remain an essential ingredient.
"But who are the humans in the loop?" people often ask, and it's a reasonable question.
If the images your algorithm is learning from were shot with a lens with a color filter, all of those images will be identically skewed. If the ruler used to determine the dimensions of data elements was ¼ inch short of a foot, all of the dimensions will be wrong by an identical proportion.
Much like with traditional software development, as AI and ML initiatives take priority on the enterprise agenda, data scientists need to retire the outdated waterfall approach and replace it a more agile style of development.
Prejudicial bias is probably what most people think of when they hear the word. And this kind of bias is the source of many scary AI headlines. And yet, this phenomenon of human prejudice influencing algorithms is complex.
In waterfall these interdependencies aren’t fully explored until the penultimate testing stage, at which point any oversights or miscalculations that are discovered drive the entire project back to the architecture or coding stage.
With sample bias in machine learning in computer vision, natural language processing and entity resolution, it isn’t possible - or even necessary - to expose an algorithm to its entire universe in order to train it. A sample of that universe is adequate, more affordable, and more practical.
ABJ notes that “CEO Nathaniel Gates expects Alegion's headcount to nearly double by the end of the year — and to pivot to becoming ‘product focused’ rather than ‘service focused’ to widen its market reach.