Really interesting. Upshot: a well trained multimodal video generation model has a world representation model trained inside it. They’ve done some work lifting this world model out and deploying it to robots, where it seems to work well.
On the one hand, this isn’t a new idea, and the quality video models certainly have understanding of materials, light, the world (at least in an Occam’s razor sense of understanding). I’m not aware of a video lab that’s turned itself into a robot lab yet, though; perhaps this would be a first, or a new sort of obvious-in-retrospect business path: train video model, sell video generation, scale, use scale to train robot things: profit.
I found their hands very interesting - looks like a bunch of stuff hidden in gloves - Xiami’s Robotics-1 foundation model just released videos of users training on some pretty standard looking grippers; to the point that there are demo videos of people putting on gripper type gloves to make video to train that model.
The BFL model looks like it doesn’t need that at all. Given the difficulty of the hardware side, I’ll be curious to see what they do with this.
What's sad to me is that we have all this awesome technology, but movies are worse than ever.
I usually watch movies from decades ago just to find something decent and it's amazing how with goofy-looking puppets the storytelling was 1000 times better.
The video at around 3.30min, where the robot arm took 3 attempts to reseat the window trim, was quite unnerving - I have not seen such resolving before. Is it new or am I way out of the loop?
You are indeed out of the loop. Google did something arguably more impressive more than a year ago, with a VLA based bot replacing a tensioned timing belt: https://www.youtube.com/watch?v=2AAFiuEP7iE
> However, compared to more specialized approaches for representation learning they produce less disentangled representations, which puts a ceiling on their usefulness for tasks that require world understanding.
Only an LLM would use a less disentangled representation of the concept “more entangled” when trying to explain to people in the real world why less disentangled representations are not as useful for modeling the real world.
We’re just pulling signs of LLM touched writing out of our ass now. Might be time to move on from the accusations, assume all writing is at least LLM assisted and judge it purely on the quality.
Bad news, humans write silly things all the time. If anything, LLMs are less likely to make awkward phrasings than people, because they aren’t found in the training data very often.
On the one hand, this isn’t a new idea, and the quality video models certainly have understanding of materials, light, the world (at least in an Occam’s razor sense of understanding). I’m not aware of a video lab that’s turned itself into a robot lab yet, though; perhaps this would be a first, or a new sort of obvious-in-retrospect business path: train video model, sell video generation, scale, use scale to train robot things: profit.
I found their hands very interesting - looks like a bunch of stuff hidden in gloves - Xiami’s Robotics-1 foundation model just released videos of users training on some pretty standard looking grippers; to the point that there are demo videos of people putting on gripper type gloves to make video to train that model.
The BFL model looks like it doesn’t need that at all. Given the difficulty of the hardware side, I’ll be curious to see what they do with this.
What's sad to me is that we have all this awesome technology, but movies are worse than ever.
I usually watch movies from decades ago just to find something decent and it's amazing how with goofy-looking puppets the storytelling was 1000 times better.
This is probably an unrelated rant, sorry
> However, compared to more specialized approaches for representation learning they produce less disentangled representations, which puts a ceiling on their usefulness for tasks that require world understanding.
Only an LLM would use a less disentangled representation of the concept “more entangled” when trying to explain to people in the real world why less disentangled representations are not as useful for modeling the real world.
if you know some one out of the software development/engineering worlds, please forward this to them
u.i. Zugló robot mikor?