Hi Pat. I'd like to think more about syndicated reference data. Particularly, with syndicated reference data, what do you think the role is for auto_increment integer primary keys? I'm thinking that syndicated reference data would prefer natural keys, such as e.g. Part Number 'ZX1234', rather than Part ID 1234 (which, being an auto_increment ID, might vary across syndicated schemata? And if Part ID is not auto_increment then how is it allocated and managed?). But if we do use natural keys for syndicated reference data, then how do we reference them within our own schema? Do we just use the natural key directly in our foreign key table, as, e.g., 'ZX1234', or should we wrap the natural key in our own table and allocate it an auto_increment ID for within our schema..? Also what processes do we have for maintaining and distributing syndicated reference data? Are hard deletions allowed or only soft deletions as with an 'is_deleted' flag? If you wanted to talk more about this I'd be happy to hear from you via your blog of you could email me at jj5@jj5.net. Hope to hear from you. Thanks!
Since teched2002 I’ve used your deck to shape the thinking of my teams around how we thinking about system boundaries and interfaces. Really glad to see this updated new version of the content
Hey Pat. Happy New Year! Sorry for the delay getting this feedback to you. I had to find some time to read your paper! One feature of your paper that I felt I should remark on are the high quality diagrams/graphics, you did a really great job with those! Following are my specific notes on Version 201228g, I'm sorry they don't add to much more than spellcheck...
line 68: s/successful/successfully/
line 100: it might be valuable to define 'fiefdom' and 'emissary' earlier, I had to look them up (they are uncommon words) before I got to the place where you defined them. No biggie. I do enjoy your fiefdom/emissary/collaboration metaphor and its entailments and elaboration.
line 103: s/am/an/
line 109: you seem to be describing corporate email. :P
line 176: this section reminds me, I used to collect paper forms. I had a folder and wherever I went if I found a paper form I would pinch a copy and keep it in my folder. I figured I was in Information Technology and forms were pretty much what it's all about. Unfortunately I lost my collection over a decade ago, and while I would like to start again from scratch I fear the world has changed: paper forms don't exist for me to collect any more!
line 226: s/for the pouring/for pouring/
line 256: s/making bank's/making the bank's/
line 320: I liked this observation of the nascent/emerging application-to-application usage of syndicated reference data.
line 326: s/since this the/since the/
line 357: I liked this notion of collaboration-based work always being offline.
line 371: s/metadate/metadata/
line 402: s/can imposed/can be imposed/
line 403: s/business's/businesses/
line 407: I also use the term 'interaction' in my designs. Basically one HTTP request or one command-line invocation is one 'interaction', and a single 'interaction' might entail multiple ACID transactions on my RDBMS. I allocate interaction IDs and log associated data.
line 423: s/cards) product-ids/cards), product-ids/
line 456: when you say "the use of an order-id or some other identifier" you express particular ecommerce leanings, but there are many Collaborations with different use cases that would fall under "some other identifier", such as Bug ID or Incident Number or Query ID etc. I know you didn't mean to preclude these use cases--in fact you went out of your way to include them--it's just that your statement got me wondering about the relative size of Order ID vs Some Other IDs that might be out there. It made me realise I have no idea about the type and scale of the systems that are out there. :P
line 512: I happened to read this page on January 12, 2021, the same date as used in the diagram! What a coincidence! :)
line 517: similarly to my comments on line 456 when I read your quantification "there may be hundreds of thousands of concurrent collaborations" it made me realise again that I am ignorant of the various scales of things. How many orders does Amazon process per day? How does this contrast to smaller independent bookstores? How many smaller independent bookstores still exist? etc.
line 560: as you raised the topic of "resource managers" I was reminded of my thinking that there seem to be two types of reasons for a change in a database. One reason is that the state of the world has changed and the database is being modified to bring it up-to-date, but the other reason, which I think is a quite different reason, is that the state of the database needs to be changed because previous data, which has been found to be erroneous, is being corrected. Having spent over 20 years designing databases and applications which use them I'm still not 100% across this distinction and how best to manage each of the two types of changes. The main problem seems to be that historical data may or may not have ever actually been the case. I wonder if there might be value in making distinctions between "updates" (to valid data) and "corrections" (to invalid data).
line 623: s/having/have/
line 629: s/manage/managed/
line 632: s/old resource/old resources/
line 711: "We had that problem..." -- Who is "we"..? Just curious... :)
Hi Pat. I'd like to think more about syndicated reference data. Particularly, with syndicated reference data, what do you think the role is for auto_increment integer primary keys? I'm thinking that syndicated reference data would prefer natural keys, such as e.g. Part Number 'ZX1234', rather than Part ID 1234 (which, being an auto_increment ID, might vary across syndicated schemata? And if Part ID is not auto_increment then how is it allocated and managed?). But if we do use natural keys for syndicated reference data, then how do we reference them within our own schema? Do we just use the natural key directly in our foreign key table, as, e.g., 'ZX1234', or should we wrap the natural key in our own table and allocate it an auto_increment ID for within our schema..? Also what processes do we have for maintaining and distributing syndicated reference data? Are hard deletions allowed or only soft deletions as with an 'is_deleted' flag? If you wanted to talk more about this I'd be happy to hear from you via your blog of you could email me at jj5@jj5.net. Hope to hear from you. Thanks!
Since teched2002 I’ve used your deck to shape the thinking of my teams around how we thinking about system boundaries and interfaces. Really glad to see this updated new version of the content
Hey Pat. Happy New Year! Sorry for the delay getting this feedback to you. I had to find some time to read your paper! One feature of your paper that I felt I should remark on are the high quality diagrams/graphics, you did a really great job with those! Following are my specific notes on Version 201228g, I'm sorry they don't add to much more than spellcheck...
line 68: s/successful/successfully/
line 100: it might be valuable to define 'fiefdom' and 'emissary' earlier, I had to look them up (they are uncommon words) before I got to the place where you defined them. No biggie. I do enjoy your fiefdom/emissary/collaboration metaphor and its entailments and elaboration.
line 103: s/am/an/
line 109: you seem to be describing corporate email. :P
line 176: this section reminds me, I used to collect paper forms. I had a folder and wherever I went if I found a paper form I would pinch a copy and keep it in my folder. I figured I was in Information Technology and forms were pretty much what it's all about. Unfortunately I lost my collection over a decade ago, and while I would like to start again from scratch I fear the world has changed: paper forms don't exist for me to collect any more!
line 226: s/for the pouring/for pouring/
line 256: s/making bank's/making the bank's/
line 320: I liked this observation of the nascent/emerging application-to-application usage of syndicated reference data.
line 326: s/since this the/since the/
line 357: I liked this notion of collaboration-based work always being offline.
line 371: s/metadate/metadata/
line 402: s/can imposed/can be imposed/
line 403: s/business's/businesses/
line 407: I also use the term 'interaction' in my designs. Basically one HTTP request or one command-line invocation is one 'interaction', and a single 'interaction' might entail multiple ACID transactions on my RDBMS. I allocate interaction IDs and log associated data.
line 423: s/cards) product-ids/cards), product-ids/
line 456: when you say "the use of an order-id or some other identifier" you express particular ecommerce leanings, but there are many Collaborations with different use cases that would fall under "some other identifier", such as Bug ID or Incident Number or Query ID etc. I know you didn't mean to preclude these use cases--in fact you went out of your way to include them--it's just that your statement got me wondering about the relative size of Order ID vs Some Other IDs that might be out there. It made me realise I have no idea about the type and scale of the systems that are out there. :P
line 512: I happened to read this page on January 12, 2021, the same date as used in the diagram! What a coincidence! :)
line 517: similarly to my comments on line 456 when I read your quantification "there may be hundreds of thousands of concurrent collaborations" it made me realise again that I am ignorant of the various scales of things. How many orders does Amazon process per day? How does this contrast to smaller independent bookstores? How many smaller independent bookstores still exist? etc.
line 560: as you raised the topic of "resource managers" I was reminded of my thinking that there seem to be two types of reasons for a change in a database. One reason is that the state of the world has changed and the database is being modified to bring it up-to-date, but the other reason, which I think is a quite different reason, is that the state of the database needs to be changed because previous data, which has been found to be erroneous, is being corrected. Having spent over 20 years designing databases and applications which use them I'm still not 100% across this distinction and how best to manage each of the two types of changes. The main problem seems to be that historical data may or may not have ever actually been the case. I wonder if there might be value in making distinctions between "updates" (to valid data) and "corrections" (to invalid data).
line 623: s/having/have/
line 629: s/manage/managed/
line 632: s/old resource/old resources/
line 711: "We had that problem..." -- Who is "we"..? Just curious... :)