[pybsddb] Batch import is slowing down

[Amirouche Boubekki]

On 2015-06-24 18:56, Jesus Cea wrote:
> On 24/06/15 12:09, Amirouche Boubekki wrote:
>> Indeed the documentation is very good, you did a very good job with 
>> that.
> 
> I didn't write it :-). In fact neither Oracle did. It was mostly 
> written
> in the SleepyCat era :), before Oracle bought them.
> 
> Anyway I am not related to Oracle in any way at all.
> 
>> - The first set of 10 000 entries took 3 seconds to load.
>> - 49th set took 2 minutes
> 
> If could be a cache size issue. I can't really tell without checking it
> myself. What does "db_stat -m" show?. Initialize the stats with 
> "db_stat
> -mZ" from time to time and compare results between the beginning and 
> the
> slow part.

I see nothing suspicious in the log. They are attached. I'll be very 
happy if you can help me.

Here are the timings:

10000 0:00:03.643761
20000 0:00:04.169674
30000 0:00:05.834176
40000 0:00:05.220522
50000 0:00:06.411978
60000 0:00:08.391819
70000 0:00:12.561121
80000 0:00:06.480804
90000 0:00:07.154480
100000 0:00:07.106232
110000 0:00:07.297861
120000 0:00:14.855432
130000 0:00:08.684627
140000 0:00:09.301576
150000 0:00:09.059745
160000 0:00:21.719725
170000 0:00:10.018946
180000 0:00:11.913997
190000 0:00:17.609332
200000 0:00:11.109638
210000 0:00:13.138263
220000 0:00:19.308328
230000 0:00:13.577793
240000 0:00:13.572754
250000 0:00:33.189006
260000 0:00:14.068787
270000 0:00:29.813695
280000 0:00:20.045663
290000 0:00:42.450155
300000 0:00:43.168721
310000 0:00:46.578547
320000 0:00:48.104117
330000 0:01:07.002841
340000 0:01:13.870403
350000 0:01:03.762506
360000 0:01:20.306731
370000 0:01:13.692656



> 
> _______________________________________________
> pybsddb mailing list
> pybsddb at jcea.es
> https://mailman.jcea.es/listinfo/pybsddb
> http://www.jcea.es/programacion/pybsddb.htm

-- 
Amirouche ~ amz3 ~ http://www.hyperdev.fr
-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: stat50k.log
URL: <https://mailman.jcea.es/pipermail/pybsddb/attachments/20150624/37ba2bf4/attachment.log>
-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: stat180k.log
URL: <https://mailman.jcea.es/pipermail/pybsddb/attachments/20150624/37ba2bf4/attachment-0001.log>
-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: stat320k.log
URL: <https://mailman.jcea.es/pipermail/pybsddb/attachments/20150624/37ba2bf4/attachment-0002.log>
-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: stat350k.log
URL: <https://mailman.jcea.es/pipermail/pybsddb/attachments/20150624/37ba2bf4/attachment-0003.log>