I think the pretext of this articles is ridiculous.
Yes, SQL is based around relational algebra, but all programming languages are built on a theoretical foundation.
And SQL is very much a "fad" language - it just somehow managed to stick around. The goal was not some sort of mathematical purity, but rather to built a natural language data interface (sounds like something currently very hyped?) and it failed spectacularly at that goal.
It is so far from natural language that English speakers with statistical understanding won't be able to read it, but it is also inconsistent enough in its grammar design that it is unreasonably difficult to learn and needs large refactoring every time you want to query into the result of a query.
To continue my rant: Sometimes '=' is an identity test, sometimes it is `==`. Sometimes groups are called groups, sometimes they are partitions.
When creating a CTE, you put the name before "AS", but when creating a column, you put the name after "AS".
SQL is great because it is everywhere and it is definitely good enough, but it is not something great, that transcends other programming languages.
> If you are a junior developer, “learn SQL properly” is the most valuable 40 hours you can spend. Not a tutorial. Not an ORM. Actual SQL: joins, subqueries, window functions, query plans. That investment pays you back at every job, in every stack, for decades
This is the power of low-level reasoning.
Today, even for a junior developers, even if they have AI that solves syntax problems, SQL teaches you to reason and approach problems logically. Without any wrapper masking low-level logic.
It's something like the letters of the alphabet that form concepts: why should they change?
>, SQL teaches you [...] Without any wrapper masking low-level logic.
I understand the point you're trying to make, and yes, it does seem like SQL is "low-level" from the perspective a wrapper like ORMs or a GUI db browser tool with menus for filtering data.
But it's also worth remembering that SQL itself is a high-level wrapper that hides the lower-level C/C++ code of the db engine that has the loops that iterate through b-trees, 8k data pages, memory blocks of the buffer cache, etc.
And C/C++ itself is a high-level wrapper that hides the logic in lower-level Linux o/s system calls that manages RAM and disk i/o.
And Linux itself is a high-level wrapper that hides low-level device drivers like SATA/SSD memory-mapped IO ... and so on and so on.
Depending on the type of app, you can ignore all the lower levels and just work at the abstraction level of higher-level wrappers.
>The Only Programming Language Built on Mathematics, Not Fashion
As a modern array language D4M is the natural successor for SQL [1].
D4M is based on mathematics like SQL, specifically associative array algebra but not relational unlike SQL. It's more generic since can it caters to most modern data abstractions including spreadsheets, database tables, matrices, and graphs [2].
You can achieve 100M database inserts per second with D4M and Accumulo more than a decade ago back in 2014 [3].
[1] D4M: Dynamic Distributed Dimensional Data Model:
Applying the Lindy effect [1]: after half a century of SQL we can expect it to survive for at least as long.
Disruption/displacement of SQL is like attempting to replace email. It's not going to happen. At best an alternative technology can carve out a small niche (and there's nothing wrong with that).
The power of SQL is not because it is "based on mathematics" - it's because anyone (really, anyone, even with the most basic English skills) could understand it quickly enough to start using it productively with not much technical knowledge. Business analytics, managers of all sorts, manual QA people could grasp the basics in a minute and more complex queries within a few hours. It is very user-friendly and such tools win over anything else. Each time I see an overengineerd/overcomplicated solution that is hard to read/understand - I know it's only "good luck" to the creators.
I've been slowly transitioning from using an ORM to just plain SQL. It's so much simpler. Less magic, more explicitness, and more control. Also, much better performance. I think the thing is to construct your model around the different queries you need to perform. In many cases, especially a CRUD-type situation, you'll end up with 10-20 different SQL queries, and that's it.
Once you break free of ORM’s I find the code so much simpler to maintain.
Here’s the query(typically multiple different subqueries and return types), here’s the params, give me all the data back and something like Dapper in .net is an absolute godsend to convert it.
- I recently read that most programmers SQL knowledge is outdated by 20 years and it’s true for me. There are quite a lot of features in most DBs that feel very "new" to me.
- Comparing SQL to React weakens the argument. SQL is the language, React is a piece of software. You certainly can run 30 year old JS today in modern browsers.
One of the best things that happened to me is my boss giving me a crash course in advanced SQL at my first job. In the database we used at work, he gave me increasingly difficult questions to answer with queries.
It was a great foundation and has served me well to this day.
It's not that I like or dislike SQL, it is just that it has such raw power and mature tooling/resources, I wonder what an alternative could even offer me.
It's like C. It does such a great job at being structured assembly that it is hard to displace it for similar reasons.
I’ve always felt that SQL is somewhat easy to grasp for basic queries, but gets complex and difficult for even moderate to higher complexity use cases. My eyes glaze over when I read long stored procedures that someone else has written. Any recommended resources to go from beginner/beginner-intermediate to advanced?
I feel like stored procedures and co crosses over into the realm of application programming, and while I can't speak from experience (so take this with a huuuuge grain of salt), this is where things break down. It feels like adding logic / basic programming to JSON/YAML, which are data/config languages primarily.
I think stored procedures - or anything that goes beyond storing / looking up data - had a place when a database had multiple different clients, but with modern day systems that's less likely to be an issue.
> Edgar Codd formalised relational algebra in 1970. SQL sits on top of it as a declarative interface. You describe what you want. The database engine decides how to get it. The engine improves every year. Your query stays the same.
Although SQL is of course not relational Algebra (and others like Datalog and D4M are better), it's still cool. It inspired kSQL like Lil uses https://beyondloom.com/decker/lil.html#lilthequerylanguage , which inspired the code I'm most proud of: https://codeberg.org/veqq/declarative-dsls A common query language, a common idiom, for many data structures (arrays, hashmaps, datafremas) is liberating, permitting you to e.g. solve sudoku, make mandelbrot sets or calculate primes directly:
(def n 40) # to reach primes up to, left is sqr of n, right n/2, then multiply them for rows
(def composites
(df/select :from (range 2 (+ 1 (math/floor (math/sqrt n))))
:cross (range 2 (+ 1 (/ n 2)))
:where |(<= (* ($ :value_left) ($ :value_right)) n)
[[:value_left :value_right] :value
|(* ($ :value_left) ($ :value_right))]))
(df/select :from (range 2 (+ 1 n)) :exclude composites)
Helps simplify complex SQL queries and no need to waste network traffic on data that client side is never going to use, and waste CPU cycles processing it.
Yes, what about database portability?
I am on my 50s and it only mattered on a single project, which was anyway a middleware for application servers.
For sure, but have a solid grounding in set theory to go with it.
I've dealt with so many poorly-performing stored procedures that ended up being written as iteration over a CURSOR when they could have been done with sets. Programmers who don't grok set theory reach for iterative constructs which, while they work fine, are an impedance mismatch with SQL.
I was a fan of Seven Languages in Seven Weeks [1] because it exposed you to different paradigms which you could then try to apply where they made sense on whatever tools you were using or building: prototype based, fault tolerante, funcional, logical. Very fun book when used right.
The point being that sometimes the tools themselves don't need to survive because you take the lessons from one thing to another (e.g. move semantics and rust/modern c++)
For me SQL has long been the gateway to the world of development. I work in the UK non-profit sector and traditionally this kind of technical knowledge is rare, so for any team I've worked with I've built learning pathways that start with SQL before pushing out into Python, Linux, and other things.
We're not exactly at the bleeding edge of current technologies, but SQL has consistently proved to be a great jumping-off point for novices who have even a passing interest in computing.
I've learned SQL around 20 years ago, and in all this time I've felt it was just a poorly designed language. It was always infuriating to write because of its verbose nature. Keywords were split into two words. I'm still shocked it's not "GROUPBY". There is no composition and modularization of logic, queries become massive expressions.
I know I'm in the minority in places like this, but I've spent all my life using ORMs, and never once regretted it. And I'm the kind of person that actually likes low-level C from time to time. SQL just feels like a poor abstraction layer: either go higher or lower.
It’s a good abstraction layer, and a fundamentally good/effecient model of organization and data management. It’s a horrible language, has a meaningless standards doc, some of the worst debugging tooling of modern system and generally any tooling outside of the RDBMS engine itself is 20 years stale.
The only difficult part in arguing this is that RDBMS != SQL != RelationalAlgebra, and it’s very often forgotten
Same can be said for learning an OO programming language or a procedural programming language. I learned C++ at school and started using Java on my first job. I forgot how to work correctly with pointers but I have tried multiple languages (using the same paradigms) and managed to build working software
Everyone knows SQL already. The harder parts that pay off are schema design, knowing how to interact with your DB in code, and knowing all the ins and outs of whatever DBMS you're using.
I would emphasize the importance of batching and set operations. This is where I think many developers lose track of the rabbit, because you don't have much control over either of these things via ORMs. You have to get your hands dirty with raw command text.
The value of this stuff is difficult to overstate. Batching allows for you to rapidly load the RDBMS. The first few times you test, it will probably go so fast you won't believe it loaded anything at all. Set operations allow for you to bring this newly loaded data to visibility in production tables nearly instantly. Your OLAP & OLTP workloads should be dominating the compute. ETL ops (loading/set ops) should be a ghost in terms of cpu time and memory. None of this is vendor specific knowledge. Every major engine has a reasonable way to bulk load and perform quick merging of records.
> I would emphasize the importance of batching and set operations.
Please, preach your gospel more loudly and frequently. It always feels like people complain about RDBMSs being slow because they run insert queries one at a time.
SQL is not a programming language. You do not write programs in SQL. It's a declarative language (or set-of-sublanguages).
> a working developer can learn once and
> use for 30 years without rewriting their mental model.
There is any number of long-living languages which satisfy this.
Plus, SQL it's not even really a single language, because the spec changes, and is huge, and few people know it fully; and the dialects have non-trivial differences; and if you switch DBMSes, you often switch SQL dialect. In that sense, it is very much like other programming languages which evolve, like C++ or Fortran or even C.
> JavaScript is an imperative language that browser wars, framework trends, and open-source maintainer preferences reshaped every few years. It rewards you for keeping up.
> Take a React component from 2015
Javascript is actually fully backwards-compatible, to not break the Web. Any javascript from 10 years ago works in the browser. This is good but also a bit of a burden, since the language can only expand but not shrink.
React is a library, and like all libraries it has breaking versions.
Not understanding the basic difference between the two kinda undermines the credibility of the article.
Also, in a similar way, core, ANSI SQL is largely backwards compatible, but all the SQL dialects linked to various DBMS implementation are generally incompatible. Obviously that's not mentioned in the article.
> Not a tutorial. Not an ORM. Actual SQL: joins, subqueries, window functions, query plans.
Not text written by a human. Not a style that an real writer would ever use. Actual AI slop: Short sentences. Incorrect facts. Not X, Y.
Love it! I am speaking as someone who has used SQL for over two decades with reasonable success. I find it extremely logical and a good fit for my mental mode. Long live SQL!!
Yes, SQL is based around relational algebra, but all programming languages are built on a theoretical foundation.
And SQL is very much a "fad" language - it just somehow managed to stick around. The goal was not some sort of mathematical purity, but rather to built a natural language data interface (sounds like something currently very hyped?) and it failed spectacularly at that goal.
It is so far from natural language that English speakers with statistical understanding won't be able to read it, but it is also inconsistent enough in its grammar design that it is unreasonably difficult to learn and needs large refactoring every time you want to query into the result of a query.
To continue my rant: Sometimes '=' is an identity test, sometimes it is `==`. Sometimes groups are called groups, sometimes they are partitions.
When creating a CTE, you put the name before "AS", but when creating a column, you put the name after "AS".
SQL is great because it is everywhere and it is definitely good enough, but it is not something great, that transcends other programming languages.
This is the power of low-level reasoning.
Today, even for a junior developers, even if they have AI that solves syntax problems, SQL teaches you to reason and approach problems logically. Without any wrapper masking low-level logic.
It's something like the letters of the alphabet that form concepts: why should they change?
I understand the point you're trying to make, and yes, it does seem like SQL is "low-level" from the perspective a wrapper like ORMs or a GUI db browser tool with menus for filtering data.
But it's also worth remembering that SQL itself is a high-level wrapper that hides the lower-level C/C++ code of the db engine that has the loops that iterate through b-trees, 8k data pages, memory blocks of the buffer cache, etc.
And C/C++ itself is a high-level wrapper that hides the logic in lower-level Linux o/s system calls that manages RAM and disk i/o.
And Linux itself is a high-level wrapper that hides low-level device drivers like SATA/SSD memory-mapped IO ... and so on and so on.
Depending on the type of app, you can ignore all the lower levels and just work at the abstraction level of higher-level wrappers.
https://modern-sql.com/
https://use-the-index-luke.com/
- https://prql-lang.org/ - https://www.malloydata.dev/
Had to reread the title again since I thought I opened a different article about TLA+.
As for SQL, if you're referring to DBMS systems, here's what E.F. Codd, inventor of relational algebra, had to say about them and the departure from his work: https://thaumatorium.com/articles/the-papers-of-ef-the-coddf...
As a modern array language D4M is the natural successor for SQL [1].
D4M is based on mathematics like SQL, specifically associative array algebra but not relational unlike SQL. It's more generic since can it caters to most modern data abstractions including spreadsheets, database tables, matrices, and graphs [2].
You can achieve 100M database inserts per second with D4M and Accumulo more than a decade ago back in 2014 [3].
[1] D4M: Dynamic Distributed Dimensional Data Model:
https://d4m.mit.edu/
[2] Mathematics of Big Data: Spreadsheets, Databases, Matrices, and Graphs:
https://direct.mit.edu/books/monograph/5691/Mathematics-of-B...
[3] Achieving 100M database inserts per second using Apache Accumulo and D4M (2017 - 46 comments):
https://news.ycombinator.com/item?id=13465141
Applying the Lindy effect [1]: after half a century of SQL we can expect it to survive for at least as long.
Disruption/displacement of SQL is like attempting to replace email. It's not going to happen. At best an alternative technology can carve out a small niche (and there's nothing wrong with that).
[1]: https://en.wikipedia.org/wiki/Lindy_effect
Here’s the query(typically multiple different subqueries and return types), here’s the params, give me all the data back and something like Dapper in .net is an absolute godsend to convert it.
- Comparing SQL to React weakens the argument. SQL is the language, React is a piece of software. You certainly can run 30 year old JS today in modern browsers.
It was a great foundation and has served me well to this day.
It's not that I like or dislike SQL, it is just that it has such raw power and mature tooling/resources, I wonder what an alternative could even offer me.
It's like C. It does such a great job at being structured assembly that it is hard to displace it for similar reasons.
I think stored procedures - or anything that goes beyond storing / looking up data - had a place when a database had multiple different clients, but with modern day systems that's less likely to be an issue.
Although SQL is of course not relational Algebra (and others like Datalog and D4M are better), it's still cool. It inspired kSQL like Lil uses https://beyondloom.com/decker/lil.html#lilthequerylanguage , which inspired the code I'm most proud of: https://codeberg.org/veqq/declarative-dsls A common query language, a common idiom, for many data structures (arrays, hashmaps, datafremas) is liberating, permitting you to e.g. solve sudoku, make mandelbrot sets or calculate primes directly:
Or e.g. The tests file has many such things (like the sudoku solver) and even datalog and minikanren implemented on top of this!That and SQLite seems to be able to scale to almost any problem, is disgustingly fast and with litestream incredibly resilient.
Helps simplify complex SQL queries and no need to waste network traffic on data that client side is never going to use, and waste CPU cycles processing it.
Yes, what about database portability?
I am on my 50s and it only mattered on a single project, which was anyway a middleware for application servers.
For sure, but have a solid grounding in set theory to go with it.
I've dealt with so many poorly-performing stored procedures that ended up being written as iteration over a CURSOR when they could have been done with sets. Programmers who don't grok set theory reach for iterative constructs which, while they work fine, are an impedance mismatch with SQL.
I have seen DBAs make wonders without changing queries, only by adding the right set of indexes.
1. C language.
2. *nix tools (shell and friends).
3. SQL.
4. Basic IPv4 networking.
These things I learned around 20 years ago, they didn't change much and they are useful for me to this day.
The point being that sometimes the tools themselves don't need to survive because you take the lessons from one thing to another (e.g. move semantics and rust/modern c++)
[1] - https://pragprog.com/titles/btlang/seven-languages-in-seven-...
I know I'm in the minority in places like this, but I've spent all my life using ORMs, and never once regretted it. And I'm the kind of person that actually likes low-level C from time to time. SQL just feels like a poor abstraction layer: either go higher or lower.
The only difficult part in arguing this is that RDBMS != SQL != RelationalAlgebra, and it’s very often forgotten
Rivalled only by Linux, shell scripts, and Cron!
The value of this stuff is difficult to overstate. Batching allows for you to rapidly load the RDBMS. The first few times you test, it will probably go so fast you won't believe it loaded anything at all. Set operations allow for you to bring this newly loaded data to visibility in production tables nearly instantly. Your OLAP & OLTP workloads should be dominating the compute. ETL ops (loading/set ops) should be a ghost in terms of cpu time and memory. None of this is vendor specific knowledge. Every major engine has a reasonable way to bulk load and perform quick merging of records.
Please, preach your gospel more loudly and frequently. It always feels like people complain about RDBMSs being slow because they run insert queries one at a time.
SQL is not a programming language. You do not write programs in SQL. It's a declarative language (or set-of-sublanguages).
> a working developer can learn once and > use for 30 years without rewriting their mental model.
There is any number of long-living languages which satisfy this.
Plus, SQL it's not even really a single language, because the spec changes, and is huge, and few people know it fully; and the dialects have non-trivial differences; and if you switch DBMSes, you often switch SQL dialect. In that sense, it is very much like other programming languages which evolve, like C++ or Fortran or even C.
Javascript is actually fully backwards-compatible, to not break the Web. Any javascript from 10 years ago works in the browser. This is good but also a bit of a burden, since the language can only expand but not shrink. React is a library, and like all libraries it has breaking versions. Not understanding the basic difference between the two kinda undermines the credibility of the article.
Also, in a similar way, core, ANSI SQL is largely backwards compatible, but all the SQL dialects linked to various DBMS implementation are generally incompatible. Obviously that's not mentioned in the article.
> Not a tutorial. Not an ORM. Actual SQL: joins, subqueries, window functions, query plans.
Not text written by a human. Not a style that an real writer would ever use. Actual AI slop: Short sentences. Incorrect facts. Not X, Y.
My brain absolutely checks out when I read this stuff now.
Not to mention that query plans are absolutely not "actual SQL".