I don’t get it. Doesn’t Atomicity guarantee that a transaction is executed to its natural end? And if the transaction is buggy, it would break application semantics regardless of what the database does. So what does C really do here?
I would like to add, that if you try to look at the combination of an application and a database managed by an database engine as a database system, then it all becomes obvious. The information system itself has a transaction, and that should be consistent. It should be consistent with the activity performed by the user, and the business rules of the organization(s) who own()s and operates the information system (aka database system).
This terminology perspective could also be extended to distributed systems (which could just be looked as system of systems).
Having to deal almost on a daily basis with the effects of eventual consistency (i.e. when some extracted some data in Excel and then compares it with something else in another Excel extracted at some different point in time, based on potentially different rules), I dream about the C letter. Unfortunately reality is not what we always want. I would also like people to be able to freely walk and talk on the streets, or play together - but this is not the current state.
Or maybe that is because we are in transition, or in other words - in a long running transaction :-) .
I have always thought that ACID is the desired set of properties of a transactional _system_ from the point of view of the user. I think of consistency as "consistent with the user's expectations", tacitly implying strong expectations. This is the sense in which we call Babe Ruth and Leo Messi consistent players.
To that end, I think of ACID as the _combined_ responsibility of the app and db, not that of the db alone, just as reliability is a whole system property; the user is not impressed if just the power supply is robust, for example. Sure, with a more powerful db, the app can let the db handle the generic parts of state management (A/I/D), and some parts of C (foreign key constraints, cascading deletes), but that is an internal matter; the user couldn't care who handles which part, as long as the money transferred as expected.
C comes across as a weak property only because we keep referring to the properties of a db taken by itself. When we take the system as a whole -- as we must -- C is a first class property, since it encodes the concept of what it means for the user's transaction to be valid -- the state change must involve all the updates expected in a transaction, in accordance with business rules.
Using an ACID database does not automatically bestow ACID to the transaction. Many apps regularly screw up atomicity by using redis in the scope of a db transaction, for example.
Let us not speak of ACID dbs. Only ACID transactional systems..
The DBMS provides durability and tools for atomicity and independence. Consistency is largely outside of the realm of DBMS developers.
The application defines consistency, and implements it using the DBMS's tools. The app chooses the transaction boundaries and locking models necessary to achieve his notion of consistency.
If the database starts out in a consistent state (again, from the point of view of the app), and each transaction is correctly consistent, then an inductive proof shows that the database remains consistent. Of course, this proof is not possible without the atomicity and independence provided by the DBMS.
In short, consistency seemed unimportant because you spent most of your career as a DBMS developer instead of an app developer. From the app's perspective, consistency and durability are everything, while atomicity and independence are merely tools for achieving consistency.
It's good that you have come to realise that Consistency is something a DBMS needs the application to take some responsibility for, foreign-key constraints etc can only get you so far, the DBMS can't model and enforce all the data integrity rules the application may need.
I think it's important to understand that Consistency is a state at the *end* of a transaction. During the transaction the database can and will be inconsistent. For example an order in a database is not consistent when only half of its items have been inserted, it is consistent only after all its items have been inserted. C.J. Date has done a lot of damage by insisting (as he does for instance on pg 415 of his book Database Design and Relational Theory) that the DBMS should enforce Consistency rules after each statement, not each commit.
The world doesn't need faster transactions (except for blockchain), it needs consistent transactions. The problem is: distributed applications do not have access the the minimum and necessary 'information' to recover over a network that can drop, reorder or duplicate packets.
I don’t get it. Doesn’t Atomicity guarantee that a transaction is executed to its natural end? And if the transaction is buggy, it would break application semantics regardless of what the database does. So what does C really do here?
I would like to add, that if you try to look at the combination of an application and a database managed by an database engine as a database system, then it all becomes obvious. The information system itself has a transaction, and that should be consistent. It should be consistent with the activity performed by the user, and the business rules of the organization(s) who own()s and operates the information system (aka database system).
This terminology perspective could also be extended to distributed systems (which could just be looked as system of systems).
Having to deal almost on a daily basis with the effects of eventual consistency (i.e. when some extracted some data in Excel and then compares it with something else in another Excel extracted at some different point in time, based on potentially different rules), I dream about the C letter. Unfortunately reality is not what we always want. I would also like people to be able to freely walk and talk on the streets, or play together - but this is not the current state.
Or maybe that is because we are in transition, or in other words - in a long running transaction :-) .
This is my take.
I have always thought that ACID is the desired set of properties of a transactional _system_ from the point of view of the user. I think of consistency as "consistent with the user's expectations", tacitly implying strong expectations. This is the sense in which we call Babe Ruth and Leo Messi consistent players.
To that end, I think of ACID as the _combined_ responsibility of the app and db, not that of the db alone, just as reliability is a whole system property; the user is not impressed if just the power supply is robust, for example. Sure, with a more powerful db, the app can let the db handle the generic parts of state management (A/I/D), and some parts of C (foreign key constraints, cascading deletes), but that is an internal matter; the user couldn't care who handles which part, as long as the money transferred as expected.
C comes across as a weak property only because we keep referring to the properties of a db taken by itself. When we take the system as a whole -- as we must -- C is a first class property, since it encodes the concept of what it means for the user's transaction to be valid -- the state change must involve all the updates expected in a transaction, in accordance with business rules.
Using an ACID database does not automatically bestow ACID to the transaction. Many apps regularly screw up atomicity by using redis in the scope of a db transaction, for example.
Let us not speak of ACID dbs. Only ACID transactional systems..
Thoughts?
The DBMS provides durability and tools for atomicity and independence. Consistency is largely outside of the realm of DBMS developers.
The application defines consistency, and implements it using the DBMS's tools. The app chooses the transaction boundaries and locking models necessary to achieve his notion of consistency.
If the database starts out in a consistent state (again, from the point of view of the app), and each transaction is correctly consistent, then an inductive proof shows that the database remains consistent. Of course, this proof is not possible without the atomicity and independence provided by the DBMS.
In short, consistency seemed unimportant because you spent most of your career as a DBMS developer instead of an app developer. From the app's perspective, consistency and durability are everything, while atomicity and independence are merely tools for achieving consistency.
It's good that you have come to realise that Consistency is something a DBMS needs the application to take some responsibility for, foreign-key constraints etc can only get you so far, the DBMS can't model and enforce all the data integrity rules the application may need.
I think it's important to understand that Consistency is a state at the *end* of a transaction. During the transaction the database can and will be inconsistent. For example an order in a database is not consistent when only half of its items have been inserted, it is consistent only after all its items have been inserted. C.J. Date has done a lot of damage by insisting (as he does for instance on pg 415 of his book Database Design and Relational Theory) that the DBMS should enforce Consistency rules after each statement, not each commit.
The world doesn't need faster transactions (except for blockchain), it needs consistent transactions. The problem is: distributed applications do not have access the the minimum and necessary 'information' to recover over a network that can drop, reorder or duplicate packets.