It just adds an extra layer of indirection, and another layer of processing that can go wrong. And when things go wrong, you have to dig down to the SQL level to understand it. But the table names are all mangled, which might further add to the complexity of debugging it.
One way that it is better is that it might be a better abstraction. If so, it would be possible to implement just these methods to target an in-memory or on-disk store without the SQL — and get a dramatic performance improvement.
But because data management requires a lot of disciplines and it is easier to test “is this is a better abstraction?” if by creating these kinds of abstractions on abstractions.
Some of them will evolve into “noSQL” but most fail because SQL has a first-movers advantage and they aren’t enough better to warrant further investment. Better may be possible but too few people can afford the luxury of exploring language design — and those that can are scared off easily; People who make bad investments worry about their fellow human and try to prevent them from making the same mistake; and so you worry about what happens when it goes wrong, which is why I say, “what if it doesn’t?”
The biggest deal for me is type safety (I use C#'s EntityFramework). While composing the ORM queries, I get the benefit of static verification, autocompletion, and can avoid the common boilerplate of joins. When reading code, find-references and so on work.
Not a java developer myself, but as a longtime coder and even an author of an ORM I can relate.
I think ORMs just bridged that gap of “how my objects look in code” and “how to store them” quite well.
But lately we’ve been removing a lot of the reasons to use ORMs altogether. By using more functional languages and features, we’ve been getting rid of that “O”. By adding more relationship features to databases like jsonb in postgres, and even nosql dbs themselves, we’ve been removing the “R” bit as well.
There’s simply a log less for ORMs left to do in modern codebases.
I’ve personally been using only raw sql with some help for building complex queries - in the JSX way of embedding stuff into the query as opposed to query builders that build it altogether, and couldn’t be happier.
With migrations handled by a separate library, all the db connection lib has to do is handle pools and run queries.
I’ve been looking into software that would statically type the sql queries themselves, which could end up as a way better dev experience.
ORMs helped there too by giving you some piece of mind that what you wrote was correct with regards to the DB schema, but that was only true if no other tool touched your db. But that is very hard for sufficiently large systems, especially in the era of microservices.
With typed sql queries it’s no longer necessary, and is even safer.
I think your post summarizes it pretty well. One more addition from my side would be to emphasize that "immutability by default" has been accepted as best practice, which does not really work well with with original idea of ORMs.
IMHO, ORM solves wrong problem. I don't see the mapping itself as a problem - in almost any language there are data structures which can represent data we query from db. The ability to map a table record to an instance of some class looks like a good idea initially. But with time you find yourself in the situation where you need to query only a subset of fields and you end up with several implicit contexts with its own set of accessible fields each.
The real problems are:
1. Boilerpalte we get in the code which extract query results. This issue is not universal. Psycopg in Python produces a lot less boilerplate than, for instance, JDBC. This problem is a lot more visible in statically typed languages.
2. Boilerplate we get creating queries. Conditionally turning part of queries on and off produces a lot of code. This is mostly due to SQL syntax being more friendly to human rather than machine. Infix logical operators in conditions, commas etc., makes it hard to write code which generates SQL.
3. SQL is not good at reusing query parts. Views don't solve all problems.
The first problem is better to solve with code generation at compile time making it possible to perform all kinds of type check etc.
Generating SQL is not that easy without introducing some kind of intermediate language which is more code friendly. I think some kind of templating could solve the problem so that it would look essntialy like SQL but with ability switch part of the query on and off.
The hardest part is reuse which I don't know how to address. For instance, adding a join to some ACL table with conditions to exclude entities the client is not authorized to access.
It’s the same as saying that wheel solves the wrong problem, because there are other way to move things. By definition, ORM is the mapping between ER and OO models. Working with ER models directly in object-oriented languages is suboptimal, because ER design addresses the problem of storage and OOD is more efficient in solving the problem of data processing. Hence the need in ORMs, which are mainstream now exactly because they enable the efficient transition from the storage to the processing data model and vice versa.
The problems which exist in modern ORMs are usually related to the complexity of their abstraction and imperfection of the programming language: the first problem is easy to notice in JPA with fetching strategies and caching, the second one is what author of this new ORM is trying to eliminate with the code generation - the need to use the ER namespace both in mapping declarations and in queries. JPA does not solve it well, quite often requiring developers to use native queries, because Criteria API is too abstract for many use cases and too heavy for modeling queries in the code. On the other side, JOOQ is not offering a convenient metamodel and operating on too low level.
Interesting, it looks like you see it in different light. Really, my point was that, first, the problem of automatic mapping doesn't need to be solved - its ok to do it manually IMO, second, I'm even unsure that OO is more efficient in data processing. This is definitely subjective, but in my experience in most cases OO didn't add up much to the clarity of my intentions.
To be honest the incentive for me to post this was mostly accumulated irritatation by the fact that a mere side effect of the mapping turned to countless attempts to model SQL in the host language. Really, IMO, all of them are worse than SQL, all of them waste your time spent learning them, with all of them sooner or later you find yourself in a position when you figure how to express the SQL you want with the DSL at hand (again, it is hard to qualify this other than wasting time).
The object-relational mapping happens everytime you fetch the data from a relational database in an object-oriented program. The moment when you try to generify your code to avoid repeating lists of columns in queries and parsing of similar result sets is the moment you create an ORM tool. All the efforts to create DSLs are coming from the idea that silver bullet exists and it’s possible to cover all use cases by a single solution. It doesn’t. Trivial ORM is enough to cover 80-90% of use cases, for the rest it doesn’t make sense to try — they are usually so unique that running plain SQL and parsing the result is the best. So I agree with you, it’s probably a waste of time to build all those DSLs when we could try to improve the simple ORM experience (Spring Data is moving in the Right direction here).
Aha, I got it now. Technically you are absolutely correct. The mapping always happens. I was just aiming at cases where original ORM intent was to provide more "smooth" OO expirience, such as having object references instead of foreign keys, 1 to 1 class to entity mapping etc.
> For instance, adding a join to some ACL table with conditions to exclude entities the client is not authorized to access.
This specific thing is better done with row-level security (in postgres at least). This requires that each user have a role in the db and that all queries be done in transactions that switch to the relevant role; but this seems to be a best practice anyway.
I cannot imagine this as a best practice, given that you will have to synchronize two identity and access management systems - the one on the frontend and the one in Postgres. Compared to ACL or filtering of results in the service tier via something like Spring Security, it’s too much overhead for such task.
As for points 1 and 2: in languages with rather powerful type-systems, this is not solved by code generation but with mere language features. Examples are Scala or D or also more academic languages like Idris or F*. No intermediate language needed.
Unfortunately many mainstream languages such as Java or C# have to rely on code generation to solve the problem.
Many Java developers say ORMs are the problem. Not so. JPA is. It is complicated, has lots of implicit behaviour, has a really bad query language and is horrible to optimize.
It is why you never hear 'ORMs are horrible' from ruby on rails or elixir developers: they have a proper ORM!
Actually Java has too, and probably several - Ebean, for example, solves all of the complexity problems, adds a query creation tool that stays very close to SQL and allows very simple object fetching, allows for simple batch operations, and requires a proper explicit save instead of the complicated implicit JPA one.
To be honest, Ecto is not an ORM, it's a query builder with superpowers. More so Elixir is a functional language and technically the O in ORM is missing. I love Ecto, I've never seen something as easy to use and maintain but it's not something Java folks would call an ORM.
True indeed, certainly not an ORM. But I would say the effect you can achieve with it is very close to what a proper ORM can do, much more so than the lower level frameworks java people are now recommending.
ORMs seem amazing up until the inevitable point of hitting a performance issue or bug, and having to print the underlying statement. At that point you are no longer fighting with the bug impacting your systems, you are fighting with the ORM midddle-ware it self, and having to quickly improve your SQL competency above everything else.
That is the moment you 've lost all the benefits advertised.
For smaller systems it might be ok I guess, but stay alive long enough and you 'll see that sooner or later every small system desires to grow, grow and grow!
the way to debug performance on a large sql system is to log things on the sql server itself and monitor for weird queries. Tjis is part of monitoring a db and this has to be done no matter which tech you used to generate those queries.
good ORMs all have a way to fine-control the underlying statements to avoid querying individual rows one by one.
I'm fairly proficient with sql queries and i still don't like writing sql queries and binding to objects manually
It might be just my experience but when using ORMs I just constantly think of how to write a SQL query in my head using the ORMs syntax. As a result I’m usually using ORMs for schema maintenance and inserts. For getting the data out of the DB SQL is a simple way to go.
I think this discussion here is good. Whether it be an ORM or something like JOOQ it is always a tradeoff. I've found that the main pain point for me with these technologies is, that in any relatively complex project, you might want to query a similar but not identical thing in different ways. For either performance or business logic reasons. So lets say you have a POJO (in the Java world), that has a bunch of annotations on it. Once you need 3 different ways to query the same thing in different contexts, that breaks down. It also hides the database layer and structure from you, that you are in fact dealing with tables, not a hierarchy of objects, which can be misleading (especially to less experienced developers). We ourselves have gone the following route: All SQL is written as SQL (We use MyBatis for interfacing with that). And the only real features we use, are mapping and flow control constructs/loops for batch data. It is a lot more writing in that sense, but it also means that the query you write is the exact same one that gets executed. As for writing some database agnostic SQL, that can be executed on all of the big 3 (Oracle, Postgres, MySQL) - I have literally never seen anyone need that in practise. It also prevents you from optimizing to database specific features (in case of postgres, CTE is a good example).
Both table-centric ORMs like JPA and query-centric Query Builders like JOOQ are popular approaches to DBMS agnostic data access. ObjectiveSQL’s main differentiator compared to JPA is a JOOQ-like fluent query builder API instead of JPQL, as far as I can see after a quick scan of the project.
If your app does not have a requirement to be DBMS agnostic then DAO (Data Access Objects) frameworks like Spring JDBC or MyBATIS might meet your needs. If you don’t like using an Internal DSL and/or embedding SQL in your code, then a YeSQL framework like HugSQL might work for you.
For a strictly typed object orient language like Java, no single data access framework style has emerged as ideal. All have to address the same issues: integrating DDL/DML into source control, supporting different DBMSes, generating DAOs, and DDL migrations.
JOOQ doesn’t seem like it removes much boilerplate. It looks like it does a ton of code generation, and then on the other side you have an extremely clunky way of getting your data in a usable format.
I don’t really have a horse in this race (in Clojure there’s no need to map between results and idiomatic data structures in your domain, and you can add as much or as little schema/type checking as you want) but in previous lives doing OOP I have never lamented them 5% of problematic SQL queries to the point I’d throw away the rest that an ORM gives me. Just break out into a more powerful query language, hide it behind an interface and you still get to return meaningful objects in your domain.
This makes sense for simple software, but jOOQ generates DB specific SQL queries depending on your DB dialect. E.g. we have a big product which supports nearly all big databases and jOOQ abstracts away the DB specific stuff wonderfully.
JOOQ's fluent API gives you safety since it is introspected from the actual DB structure. Invalid code does not compile, instead of blowing up in production at runtime as it would in the case of the SQL string.
The problem is: what is the single source of truth (TM) for the expected DB schema? An ORM will (hopefully) manage a meta table with all the information. If I work on a collaborative project, I might rely on an outdated table structure. So ideally, the app reflects the DB schema in some type-safe manner.
There is no problem and the answer has already been giving: the migration library is the source of truth for the schema and that's it. There is just no ORM and if there is one it should not even have the permissions to execute any schema changes.
> So ideally, the app reflects the DB schema in some type-safe manner.
That's right! But there is no need to use an ORM for that. JooQ and other libraries (in my world it would be Quill https://getquill.io/) do that for you.