My field is primarily audio/vision ML so I don't have any particular domain expertise, but I don't get how you can lump all "time-series" together in one big bucket. My naive understanding is that the way to improve the accuracy is to gain as much joint information as possible. Like:
> Known-future covariates. Some drivers of sensor behaviour are known in advance, such as production schedules, planned shutdowns, weather forecasts. Timestral can condition on them to improve forecast quality, so a scheduled load increase is expected rather than flagged as a surprise.
To me this should be the entire basis of the system, not a condition to improve forecast quality? Like if you're modelling electricity consumption, if all you have is a window of N previous samples, you're going to get completely beaten by a comparatively simple system that knows the time of day, that there's holidays coming up, the weather forecast, etc.
I would also think that these time-series forecasting models would gain from a sophisticated understanding at least the labels of the covariates? A 5M model has no idea what a "bank holiday" is, it just sees an arbitrary class go from 0 to 1, and won't have any useful inductive biases as to what it means or how to link them to other pieces of information.
E.g., wouldn't you predict €/MWh of electricity far more accurately with access to price of natural gas (itself influenced by many geopolitical factors), breakdown in composition of production of each energy source, predictions about roll-out (requiring a sophisticated LLM capable of understanding news and their implications), etc. What's the use in trying to predict it from 2 variables and a narrow window looking back at just 8192 or 32768 samples?
Or perhaps I misunderstand the use-case and this is more for extremely short term movements?
For some kinds of forecasting it doesn't need to be exactly accurate - eg say you are forecasting traffic for how to scale a load balancer, it can be a bit off it will just cost you a little and you can adjust after - a lot of feedback systems dont need full accuracy. You wouldnt use this to forecast whether to buy shares. It basically replaces various traditional curve fitting tools. If you need a really good forecast yes you are going to build a domain model. At least this is my understanding...
You're right but, if the time series looks like music then you can apply your music domain knowledge. You don't have to only apply knowledge that's general across every domain, part of the magic of large neural nets is that you can do recognition and prediction at the same time thereby using (statistically) things known from training about how to identify a sequence and then also how to continue it.
Sure adding outside info helps even more, but with enough data it should become unnecessary (or so goes the theory).
Any 'vibe check' on these; I don't have benchmarks but I tried out Le Chonk for general coding work that they claim it works really well at and I was very disappointed, it seemed to make all kinds of obvious errors of the kind that I haven't seen from opus, sol, deepseek flash etc in over a year
I've been using Mistral Large 4 and some of my other friends have as well.
Something is going on with the performance (as in speed of the service, not quality). Some days it's quite fast, some days it's so slow I interrupt it because I thought it was hanging.
I think they're shifting hosting capacity around, or otherwise messing with the model. Note that it's still a "preview" version, meaning they're still training it further for the final release and they are likely still figuring out the optimal hosting configuration as well.
It wouldn't surprise me to see further changes in both quality of the model and speed before the final release.
Very nice. It’s mind-blowing to me that it used to take me months to build and deploy a forecasting model five years ago. Now in minutes I can use something like this, implemented with help from an LLM, and get apparently quite good results (and fine-tune on my data, it seems).
Speed looks nice too. I want an open weight model with this performance now.
Ah thanks!
The Mistral model has a restrictive license, where you need a specific commercial agreement to use it if you make more than 20M$ revenue/month. Ours is Apache 2.
If you compare the 2, it's not a strict "timestral is better" story - the winrates are tight - your probably should try both on your use-case. We have a 42% winrate vs Timestral on fevbench for instance.
Ofc, our main value is not just these models, but also 1) extra capabilities (like our model that can natively account for stockouts/censored data, and others coming soon), 2) our systems and products around the model, and 3) we're all laser focused on time-series/forecasting
Meh....time series data is fundamentally different from textual data. I see why a time series "foundation model" seems appealing, especially to the marketing folks at Mistral. Real-world usability is very limited, which is probably the reason why the blog post features an imaginary customer rather than an actual reference.
Time based data is extremely important, and there are tons of techniques out there to train super fast super small models, but they are a pain in the ass to tune. I could see a foundation model that is kind of good at a lot of things, that could be fine tuned or used when you want something more straight forward to use to analyze a bunch of data.
Honestly this might be very good for feature selection to refine a much more specialized model.
Cofounder of The Forecasting Company here - disagree on this. The models now work quite well and are in prod at a number of places, including very sophisticated places - quant funds, top retail cos etc
You are telling me quant funds prefer a heavy-weight and slow ANN to a fast, specialized model? Maybe in totally non-critical environments, otherwise it's simply too slow, regardless of performance.
> Known-future covariates. Some drivers of sensor behaviour are known in advance, such as production schedules, planned shutdowns, weather forecasts. Timestral can condition on them to improve forecast quality, so a scheduled load increase is expected rather than flagged as a surprise.
To me this should be the entire basis of the system, not a condition to improve forecast quality? Like if you're modelling electricity consumption, if all you have is a window of N previous samples, you're going to get completely beaten by a comparatively simple system that knows the time of day, that there's holidays coming up, the weather forecast, etc.
I would also think that these time-series forecasting models would gain from a sophisticated understanding at least the labels of the covariates? A 5M model has no idea what a "bank holiday" is, it just sees an arbitrary class go from 0 to 1, and won't have any useful inductive biases as to what it means or how to link them to other pieces of information.
E.g., wouldn't you predict €/MWh of electricity far more accurately with access to price of natural gas (itself influenced by many geopolitical factors), breakdown in composition of production of each energy source, predictions about roll-out (requiring a sophisticated LLM capable of understanding news and their implications), etc. What's the use in trying to predict it from 2 variables and a narrow window looking back at just 8192 or 32768 samples?
Or perhaps I misunderstand the use-case and this is more for extremely short term movements?
You're right but, if the time series looks like music then you can apply your music domain knowledge. You don't have to only apply knowledge that's general across every domain, part of the magic of large neural nets is that you can do recognition and prediction at the same time thereby using (statistically) things known from training about how to identify a sequence and then also how to continue it.
Sure adding outside info helps even more, but with enough data it should become unnecessary (or so goes the theory).
With such strong human capital, both in quantity and quality, Europe should in theory be competitive with US and China in AI.
Edit : working with GLM5.3 now, it's also slow. Maybe Mistral's under heavy load.
Disclaimer : Not Cofounder of The Forecasting Company here
Something is going on with the performance (as in speed of the service, not quality). Some days it's quite fast, some days it's so slow I interrupt it because I thought it was hanging.
I think they're shifting hosting capacity around, or otherwise messing with the model. Note that it's still a "preview" version, meaning they're still training it further for the final release and they are likely still figuring out the optimal hosting configuration as well.
It wouldn't surprise me to see further changes in both quality of the model and speed before the final release.
Speed looks nice too. I want an open weight model with this performance now.
EDIT: ours is Apache 2, whereas Timestral has a restrictive license
If you compare the 2, it's not a strict "timestral is better" story - the winrates are tight - your probably should try both on your use-case. We have a 42% winrate vs Timestral on fevbench for instance.
Ofc, our main value is not just these models, but also 1) extra capabilities (like our model that can natively account for stockouts/censored data, and others coming soon), 2) our systems and products around the model, and 3) we're all laser focused on time-series/forecasting
Seems unprofessional to me, you already started dunking on this release but added little to no arguments as to why or what is wrong here.
https://www.theforecastingcompany.com/blog/t0-gamma/
https://github.com/Nixtla/nixtla
Honestly this might be very good for feature selection to refine a much more specialized model.