Sick of sorting bricks? So was 25-year-old Daniel West, until his skill and determination brought him to build the incredible "Universal Lego Sorter". Made from 10,000+ Lego bricks itself, along with a bunch of other mechanical parts, the machine utilises deep learning AI to sort "any brick that has ever been produced" into one of 18 output buckets, at a rate of one brick every two seconds.
While it's not the first automated Lego sorter, or even the first to use AI, West's project is distinct in that the AI has been trained using 3D renders of Lego parts. So even if it's never seen a part in real life, it is still able to recognise and categorise it.
Here's how it works.
First, add Lego! The input bucket can hold a large volume of Lego ready to be fed into the mechanism. Once its been fed, the pieces are separated by a series of belts that push lines of parts onto the "vibration feeder." This segment prevents parts from sitting on top of one another, moving them steadily along to the next section.
Once separated, the parts are dropped one by one into the enclosed, brilliantly lit, white-bricked scanner section, where a Raspberry Pi Camera Module V2 records streaming video footage of each part. The footage is processed into a series of images by the Raspberry Pi 3 Model B+, and these images are finally relayed to a more powerful computer for categorisation.
Okay, so this means the process is not fully powered by the Raspberry Pi, but you try running a neural network on Raspberry Pi. Chatting to PC Gamer, West admitted the use of an RTX 2060 laptop was a little overkill and noted "it would probably be possible to use something like the Jetson Nano to run the machine entirely self-contained."
The AI algorithm we're talking about is the go-to for sorting machines. It utilises a convolutional neural network, basically meaning it's a super powerful tool for the recognition and classification of objects. And having been trained with 3D models of Lego bricks, as long as there exists a 3D model of the brick it's being shown, it should be able to classify it. No problem.
Just don't go feeding it lint balls, chewing gum wrappers, or more of the usual crap you find in a Lego hoard—who knows how it would react.
West goes into more detail here about the process of teaching the AI.
The RC buggy motor used for the vibration feeder was probably to most awkward part to get hold of, West tells us, everything else was pretty commonly available. And it sure looks like he had no shortage of Lego parts lying around to build with.
Best VR headset: which kit should you choose?
Best graphics card: you need serious GPU power for VR
Best gaming laptop: don't get tied to your desktop in VR
The project took around three years to finalise, and wasn't without its hurdles. West recalls the "reality gap" issue that surrounds using synthetic, fake data such as 3D models to train machines. "It ends up performing terribly in real world conditions, much worse than you'd expect," he explains, "it took years of failed experiments before I figured out how to crack it."
He admits he was even ready to abandon the project at one point, but we're all really glad he didn't. Just goes to show where a little perseverance can get you.
Next on the agenda for West, is "more sorter-related work I can't talk about (yet)," and its likely he'll be stepping back into the game development fray soon, too. In fact, we've written about West's game development exploits previously on PC Gamer, so take a look if you want more of a peek into the eclectic mind we're dealing with here.
As always, keep building.
