[pybsddb] Bulk Data load into a DB

[hasanyasin]

Reading this may help: 
http://www.oracle.com/technology/documentation/berkeley-db/db/programmer_reference/am_misc_bulk.html

Please share your test results.

On 28.03.2010 01:14, hasanyasin wrote:
> Another try: Using strings for key and value with DB_MULTIPLE flag 
> throws the exception traced below. So, maybe we can do it if we can 
> learn how to provide the method a "bulk key buffer".
>
> Traceback (most recent call last):
>   File "<stdin>", line 1, in <module>
> bsddb3.db.DBInvalidArgError: (22, 'Invalid argument -- DB->put with 
> DB_MULTIPLE(_KEY) requires a bulk key buffer')
>
> On 28.03.2010 01:10, hasanyasin wrote:
>> Flags are integers in fact and we operate "bitwise or" ie. | 
>> operator, on them to create one integer. So, instead of writing 
>> db.DB_BTREE, db.DB_CREATE you need to write db.DB_BTREE|db.DB_CREATE. 
>> The code you provided at your mail would work but not as aimed.
>>
>> I DB_MULTIPLE is defined in module bsddb3.db, so in order to use it 
>> you need to type db.DB_MULTIPLE in your case since you have imported 
>> db from bsddb3.
>>
>> The real exception we get here is TypeError complaining as: "string 
>> or integer object expected for key, tuple found." So, it seems 
>> current interface does not yet support this feature.
>>
>> hasanyasin
>>
>>
>> On 27.03.2010 20:41, Joepaul Puthenangadi wrote:
>>> Sorry about that. Should have been more verbose. My bad.
>>>
>>> ############
>>> This is what I did initially
>>>
>>> from bsddb3 import db
>>>
>>> db1 = db.DB()
>>>
>>> db1.open("Temp.db", None, db.DB_BTREE, db.DB_CREATE)
>>>
>>> db1.put('abc%200105%Key1','jp1p')
>>> db1.put('abc%200105%Key2','jp2p')
>>>
>>> db1.close()
>>>
>>> Worked  great
>>> ############
>>> Then, was trying to add more keys to it in  the bulk format. And was 
>>> trying to think that a tuple of keys and tuple of values in one put 
>>> will be the way to get the bulk data loaded. That was the code 
>>> pasted earlier. This is command i used from the last post
>>>
>>> db1.put(keys, val, None, DB_MULTIPLE)
>>>
>>> Then you get this error message
>>> NameError: name 'DB_MULTIPLE' is not defined
>>>
>>> So my thought was may be this feature is still not implemented. Just 
>>> want to see what members here thought.
>>>
>>> Appreciate all your help and discussions.
>>> Thanks
>>>
>>>
>>> --- On *Sat, 3/27/10, hasanyasin /<hasanyasin at cubicpen.com>/* wrote:
>>>
>>>
>>>     From: hasanyasin <hasanyasin at cubicpen.com>
>>>     Subject: Re: [pybsddb] Bulk Data load into a DB
>>>     To: "Python bindings for Oracle Berkeley DB" <pybsddb at jcea.es>
>>>     Date: Saturday, March 27, 2010, 1:57 AM
>>>
>>>     Hi,
>>>     I am not sure if that feature is interfaced in pybsddb yet; but
>>>     before that, I will ask you some other things: When you are
>>>     opening a db, it would be better to provide dbtype and flags so
>>>     we can see what the dbtype and flags of it are. Also, if that db
>>>     did not exist already, it won't open without DB_CREATE flag. The
>>>     name "DB_MULTIPLE" is not importad to global scope so you can
>>>     access it via "db" module name as db.DB_MULTIPLE.
>>>
>>>     In addition to these, saying "this didn't work" does not give
>>>     any clues to us. Please provide the exceptions throwed.
>>>
>>>     Have a good day,
>>>     hasanyasin
>>>
>>>     On 27.03.2010 00:38, Joepaul Puthenangadi wrote:
>>>>     Hi there,
>>>>
>>>>     Was reading through the documentation and found something on
>>>>     the oracle website about the possibility of loading bulk key
>>>>     and values into a hash or btree db using the flag =
>>>>     |DB_MULTIPLE. Not trying to be picky or anything just wanted to
>>>>     know whether that is something yet to be implemented to |bsddb3
>>>>     module.?
>>>>
>>>>     Again thanks a lot for the wonderful module.
>>>>
>>>>     Thanks again
>>>>
>>>>     #Sample code
>>>>     |
>>>>     |from bsddb3 import db
>>>>
>>>>     db1 = db.DB()
>>>>     db1.open("Temp.db")
>>>>
>>>>     keys = ('key1','key2','key3')
>>>>     val = ('10','15','20')
>>>>
>>>>     db1.put(keys, val, None, DB_MULTIPLE)
>>>>
>>>>     db1.close()
>>>>
>>>>     ###
>>>>     This did not work. Any thoughts.?
>>>>
>>>>     Thanks
>>>>
>>>>
>>>>
>>>>     _______________________________________________
>>>>     pybsddb mailing list
>>>>     pybsddb at jcea.es
>>>>     https://mailman.jcea.es:28443/listinfo/pybsddb
>>>>     http://www.jcea.es/programacion/pybsddb.htm
>>>
>>>     -----Inline Attachment Follows-----
>>>
>>>     _______________________________________________
>>>     pybsddb mailing list
>>>     pybsddb at jcea.es </mc/compose?to=pybsddb at jcea.es>
>>>     https://mailman.jcea.es:28443/listinfo/pybsddb
>>>     http://www.jcea.es/programacion/pybsddb.htm
>>>
>>>
>>>
>>> _______________________________________________
>>> pybsddb mailing list
>>> pybsddb at jcea.es
>>> https://mailman.jcea.es:28443/listinfo/pybsddb
>>> http://www.jcea.es/programacion/pybsddb.htm
>>
>>
>> _______________________________________________
>> pybsddb mailing list
>> pybsddb at jcea.es
>> https://mailman.jcea.es:28443/listinfo/pybsddb
>> http://www.jcea.es/programacion/pybsddb.htm
>
>
> _______________________________________________
> pybsddb mailing list
> pybsddb at jcea.es
> https://mailman.jcea.es:28443/listinfo/pybsddb
> http://www.jcea.es/programacion/pybsddb.htm
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mailman.jcea.es/pipermail/pybsddb/attachments/20100328/5f3029d1/attachment.htm>