[pybsddb] How to manage logs

[Jesus Cea]

On 18/06/15 16:02, Lauren Foutz wrote:
>> It will create one transaction per operation (get, put, delete).
> 
> Yes.
> 
>> Does it provide any speed over using transaction explicitly?
> 
> No, it tends to be slower since each commit requires that the logs be
> flushed to disk.  It is better to use an explicit transaction, and use
> the same transaction over multiple put/get/delete operations.

You have two choices.

The first choice, as Lauren says, is to do several unrelated changes in
a single transaction.

Other option I would recommend in this particular situation (massive
loading of initialization data) is to use only ACI instead of ACID: do
not force a flush to the disk after every transaction. If your computer
crashes, your database will be stable and safe but you could lose the
more recent transactions. I think this is very appropriate for this
particular situation. After you populate the database go back to ACID
and enjoy the power of a real database.

<https://docs.oracle.com/cd/E17076_04/html/api_reference/C/txnbegin.html>

Check flags "DB_TXN_NOSYNC" and "DB_TXN_WRITE_NOSYNC".

Without one of those flags a transaction commit forces a disk sync. This
is typically limited to 100 transactions per second with a costumer
grade harddisk (very dependent of the filesystem used). Skipping the
sync you would drop the D in ACID and get only ACI, but performance will
be not limited by the harddisk transaction speed.

When you do a checkpoint, Berkeley DB will do a full harddisk flush if
necessary.

-- 
Jesús Cea Avión                         _/_/      _/_/_/        _/_/_/
jcea at jcea.es - http://www.jcea.es/     _/_/    _/_/  _/_/    _/_/  _/_/
Twitter: @jcea                        _/_/    _/_/          _/_/_/_/_/
jabber / xmpp:jcea at jabber.org  _/_/  _/_/    _/_/          _/_/  _/_/
"Things are not so easy"      _/_/  _/_/    _/_/  _/_/    _/_/  _/_/
"My name is Dump, Core Dump"   _/_/_/        _/_/_/      _/_/  _/_/
"El amor es poner tu felicidad en la felicidad de otro" - Leibniz

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 473 bytes
Desc: OpenPGP digital signature
URL: <https://mailman.jcea.es/pipermail/pybsddb/attachments/20150621/4221489d/attachment.asc>