Pretty clever to keep the existing parquet files as is (assuming daily partitioned) to not break any analytical queries. Also creating this index for the first time would make me sweat. With exabytes in the lake, the backfill bill will be $$$ (even just scanning the key column).
I'm not sure about the AI agent angle though, why would it need that specific context on a user level? I guess they are exploring a "chat with my spotify history" or user generated playlists?
(also the article has several LLM smells. It's technically interesting but reads like slop at times)
I'm not sure about the AI agent angle though, why would it need that specific context on a user level? I guess they are exploring a "chat with my spotify history" or user generated playlists?
(also the article has several LLM smells. It's technically interesting but reads like slop at times)