[Libpqxx-general] Two phase commit

[Jeroen T. Vermeulen]

On Sat, August 15, 2009 17:59, Maurice Gittens wrote:

> Sorry, the late follow up. I seem to have missed your reply.

Actually it was me who replied late!  But I'm glad to have this
opportunity for some good discussion about 2pc, so I'll try not to let it
slide again.


>> Unfortunately there's no support for tpc yet.  Something I've wanted to
>> add for a long time, really.
>
> Can you give a sketch of the design and the API's you have in mind?
> Maybe, it is doable without too much work.

Actually the API design is what I haven't done yet.  :-)

The most obvious way to go would be to introduce a new transaction method
"prepare(id)" that brings the transaction into a new state where it's
detached from the connection and won't accept any further commands except
commit() or abort().  It would be possible to start new transactions in
the meantime.  Changes made by the transaction would "disappear" again
between prepare() and commit().  Both commit() and abort() would issue
different SQL in the prepared state than before it.

On nontransactions, prepare() would be either a no-op or an error.  And
using 2pc with robusttransaction would be redundant, so that leads us to a
design that offers the prepare() method only in some new transaction type.
 We could even use the transaction's name as its id.  The composition of
transaction classes becomes a bit more complex internally, but the
external API would be nice and clean.

However this is a pure two-tier design, completely non-transparent to the
application.  To me, two-tier 2pc is an invitation to disaster.

ISTM that ideally a transaction manager would be able to "inject" itself
into the application, with the application continuing to begin and commit
transactions as if it were still in direct control.

Maybe that's a pipe dream since the app should know that it's talking to
multiple resource managers anyway; or maybe complex apps could have
modules each with their own transactional "contracts" that need to be
encapsulated transparently.

Here's what I think a transparent 3-tier model would require:
 * A transaction factory to hide the difference in transaction types.
 * Obtaining a transaction id from the transaction manager.
 -> Can we do this in the factory, at the start of the transaction?
 * commit() actually prepares, then waits as the TM does the commit.

Talking to the transaction manager requires configuration, and possibly
even pluggable protocol support.

Neither of these two models can be cleanly and easily implemented on top
of the other.  The ways I see of doing that would be:

(a) go with the 3-tier model but provide a "fake" transaction manager to
give the program direct control of its own 2pc, or

(b) go with the 2-tier model except prepare() is still called commit(),
and expect the application to implement 3-tier or its own cleanup on top. 
Semantics for commit() would be radically unusual, so the transactor
framework would have to learn about this as well.

Thoughts welcome!  Transaction management is a big world that I haven't
kept much track of.  Maybe current standards force once choice or the
other, though I'm not expecting any final solution to have arrived.


> So basically, if I use non-transactions for everything, then I can simply
> issue the
> SQL  statements directly to the underlying connection?

Yes, you can.  It's not generally recommended, because the library should
support the structure you need, but this is really prototyping for the
right feature.


> Hmm, this sounds tempting, since I can use SQL statements for transaction
> management and
> libpqxx functionality for everything else, Right?

Yes.  Some more specialized features such as tablestreams may think you're
not in a real transaction and refuse to work, but basic functionality
should work.  For the more specialized stuff some unrelated errors may be
misinterpreted as "you can't do this right now, you're not in a
transaction" by the library.


Jeroen