[pybsddb] Bulk Data load into a DB

[hasanyasin]

Another try: Using strings for key and value with DB_MULTIPLE flag 
throws the exception traced below. So, maybe we can do it if we can 
learn how to provide the method a "bulk key buffer".

Traceback (most recent call last):
   File "<stdin>", line 1, in <module>
bsddb3.db.DBInvalidArgError: (22, 'Invalid argument -- DB->put with 
DB_MULTIPLE(_KEY) requires a bulk key buffer')

On 28.03.2010 01:10, hasanyasin wrote:
> Flags are integers in fact and we operate "bitwise or" ie. | operator, 
> on them to create one integer. So, instead of writing db.DB_BTREE, 
> db.DB_CREATE you need to write db.DB_BTREE|db.DB_CREATE. The code you 
> provided at your mail would work but not as aimed.
>
> I DB_MULTIPLE is defined in module bsddb3.db, so in order to use it 
> you need to type db.DB_MULTIPLE in your case since you have imported 
> db from bsddb3.
>
> The real exception we get here is TypeError complaining as: "string or 
> integer object expected for key, tuple found." So, it seems current 
> interface does not yet support this feature.
>
> hasanyasin
>
>
> On 27.03.2010 20:41, Joepaul Puthenangadi wrote:
>> Sorry about that. Should have been more verbose. My bad.
>>
>> ############
>> This is what I did initially
>>
>> from bsddb3 import db
>>
>> db1 = db.DB()
>>
>> db1.open("Temp.db", None, db.DB_BTREE, db.DB_CREATE)
>>
>> db1.put('abc%200105%Key1','jp1p')
>> db1.put('abc%200105%Key2','jp2p')
>>
>> db1.close()
>>
>> Worked  great
>> ############
>> Then, was trying to add more keys to it in  the bulk format. And was 
>> trying to think that a tuple of keys and tuple of values in one put 
>> will be the way to get the bulk data loaded. That was the code pasted 
>> earlier. This is command i used from the last post
>>
>> db1.put(keys, val, None, DB_MULTIPLE)
>>
>> Then you get this error message
>> NameError: name 'DB_MULTIPLE' is not defined
>>
>> So my thought was may be this feature is still not implemented. Just 
>> want to see what members here thought.
>>
>> Appreciate all your help and discussions.
>> Thanks
>>
>>
>> --- On *Sat, 3/27/10, hasanyasin /<hasanyasin at cubicpen.com>/* wrote:
>>
>>
>>     From: hasanyasin <hasanyasin at cubicpen.com>
>>     Subject: Re: [pybsddb] Bulk Data load into a DB
>>     To: "Python bindings for Oracle Berkeley DB" <pybsddb at jcea.es>
>>     Date: Saturday, March 27, 2010, 1:57 AM
>>
>>     Hi,
>>     I am not sure if that feature is interfaced in pybsddb yet; but
>>     before that, I will ask you some other things: When you are
>>     opening a db, it would be better to provide dbtype and flags so
>>     we can see what the dbtype and flags of it are. Also, if that db
>>     did not exist already, it won't open without DB_CREATE flag. The
>>     name "DB_MULTIPLE" is not importad to global scope so you can
>>     access it via "db" module name as db.DB_MULTIPLE.
>>
>>     In addition to these, saying "this didn't work" does not give any
>>     clues to us. Please provide the exceptions throwed.
>>
>>     Have a good day,
>>     hasanyasin
>>
>>     On 27.03.2010 00:38, Joepaul Puthenangadi wrote:
>>>     Hi there,
>>>
>>>     Was reading through the documentation and found something on the
>>>     oracle website about the possibility of loading bulk key and
>>>     values into a hash or btree db using the flag = |DB_MULTIPLE.
>>>     Not trying to be picky or anything just wanted to know whether
>>>     that is something yet to be implemented to |bsddb3 module.?
>>>
>>>     Again thanks a lot for the wonderful module.
>>>
>>>     Thanks again
>>>
>>>     #Sample code
>>>     |
>>>     |from bsddb3 import db
>>>
>>>     db1 = db.DB()
>>>     db1.open("Temp.db")
>>>
>>>     keys = ('key1','key2','key3')
>>>     val = ('10','15','20')
>>>
>>>     db1.put(keys, val, None, DB_MULTIPLE)
>>>
>>>     db1.close()
>>>
>>>     ###
>>>     This did not work. Any thoughts.?
>>>
>>>     Thanks
>>>
>>>
>>>
>>>     _______________________________________________
>>>     pybsddb mailing list
>>>     pybsddb at jcea.es
>>>     https://mailman.jcea.es:28443/listinfo/pybsddb
>>>     http://www.jcea.es/programacion/pybsddb.htm
>>
>>     -----Inline Attachment Follows-----
>>
>>     _______________________________________________
>>     pybsddb mailing list
>>     pybsddb at jcea.es </mc/compose?to=pybsddb at jcea.es>
>>     https://mailman.jcea.es:28443/listinfo/pybsddb
>>     http://www.jcea.es/programacion/pybsddb.htm
>>
>>
>>
>> _______________________________________________
>> pybsddb mailing list
>> pybsddb at jcea.es
>> https://mailman.jcea.es:28443/listinfo/pybsddb
>> http://www.jcea.es/programacion/pybsddb.htm
>
>
> _______________________________________________
> pybsddb mailing list
> pybsddb at jcea.es
> https://mailman.jcea.es:28443/listinfo/pybsddb
> http://www.jcea.es/programacion/pybsddb.htm
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mailman.jcea.es/pipermail/pybsddb/attachments/20100328/10c271be/attachment.htm>