World’s first AI-powered LEGO sorting machine built with 10,000 LEGO bricks [Exclusive Feature]

There are some individuals who find sorting LEGO pieces therapeutic, but most of us loathe the task. And there are entirely non-LEGO machines that could do it, although what fun is that? Some people have tried to use LEGO to build sorting machines, yet their limitations have been quickly apparent. Enter Daniel West and his incredible Universal LEGO Sorting Machine! This baby uses Artificial Intelligence, with the most extensive index to date, to sort parts at a speedy one brick every 2 seconds!

What makes this sorting machine so unique is that it utilizes a concept called a Convolutional Neural Network. It uses existing databases to learn and recognize nearly all pieces ever produced, even positioned different ways. That’s over 25 million 3D images! What databases, you ask? They’re ones that you may use on a regular basis, such as the LDraw parts library and Rebrickable. We asked Daniel about some other aspects of the project and the numbers are certainly impressive. He estimates that the build uses roughly 10,000 LEGO elements, including six LEGO motors. It also uses several non-LEGO parts, including nine servo-motors and a Raspberry Pi brain. It’s split into three modules — part separation, scanning, and bucket distribution.

The build process and concept design

As Daniel tells us, the inspiration came in 2011 after seeing other sorting machines on Youtube, but work didn’t start until 2016 while he was studying “computer vision” at university. Unfortunately, early tests didn’t work, until a year later when he realized he should implement AI. The building portion took about 6 months, with many iterations by trial and error.  The most difficult parts to incorporate were the vibrating feeder and output buckets. But the real triumph was the programming development, which took a whopping 2.5 years! For the programming geeks out there: Daniel used Python to write the code, as well as Tensorflow for the machine learning framework. Processing 25 million images also required a huge amount of computing power: Daniel used Amazon Web Services (AWS) to perform 2 years of CPU core time in just over a day!

Some of you might have an excellent grasp on what that is, but for the laypeople out there, Daniel has a video to explain it more simply.

While building this marvelous machine, Daniel tells us that he realized these complex problems were nearly impossible to solve with LEGO-produced electronics. The possibilities of LEGO may be virtually endless, but this required some more serious technology. For example, vivid lighting, a high-resolution camera, and a special conveyor belt were all necessary. Additionally, the cost of using LEGO servo motors and Mindstorms brains would’ve been too expensive. Daniel also says that his goal was actually not to build the machine out of LEGO, but that LEGO lent itself perfectly to some of the structure.

Moving forward

Daniel is continuing to fine-tune his work. He has already written more than one article about the project on Toward Data Science, and he hopes to also write an academic paper on the topic. When asked about drawing up early building plans, he tells The Brothers Brick that he didn’t use any and says, “I think one of the key benefits of working with LEGO is that redesigning or changing the shape of something is so nondestructive, it allows you to be really flexible and agile when it comes to design.” It’s uncertain if he will offer building instructions to people who want one of their own, but the good news is that he hopes to turn the programming into an open-source dataset! He’s excited to see what others come up with. It’s safe to say that his advice to builders attempting something similar would be to go for it and don’t give up.


Although it’s less advanced, check out another impressive LEGO sorting machine built by the BrickIt team using only LEGO. Or if you’re stuck with doing it the old-fashioned way, consider this sorting tips essay. And if you’re a fan of both LEGO and AI (if you’re here, of course you are!) check out Braille LEGO and AI.

3 comments on “World’s first AI-powered LEGO sorting machine built with 10,000 LEGO bricks [Exclusive Feature]

  1. Chris

    He mentioned Jacques’ machine (and another, even earlier one) in his video, but says this machine is the next evolution on the concept.

    The part I’m curious about is what his sort criteria are – the bins looked like they were full of a pretty wide variety of parts, and it looked like the same parts in adjacent bins. He’s obviously not sorting by color, but what are the 18 groupings?

    And could it be taught to sort by color after the initial sort? Like sort into 18 groups, then take each group and sort into 18 subgroups, and so on, until the final step would be sorting by color. 18 groups x 18 subgroups x 18 sub-subgroups is 5,800+ parts!

Comments are closed.