more results from the cluster - 5 billion triples
The gods have smiled upon us and given us a bit more time on this cluster.
It seems that Bryan has the RAM problem on the clients solved - they are no longer swapping. This has let us run out to a different problem - RAM demands on the data services. :-)
Good progress though, last night's run yielded 5 billion triples loaded in just under 10 hours for an average throughput of 135k triples per second. Max throughput was just above 210k triples per second. 1 billion triples was reached in an astonishing 78 minutes.
Configuration was 20 data services (10 blades with 2 data services each), 8 client services (4 blades with 2 each), and one blade for centralized services.
Note that these times are for simple RDF load - no closure. Previous tests have demonstrated that closure takes about as long as simple load - so double the time and halve the throughput to get our numbers with closure. Still quite impressive.
It seems that Bryan has the RAM problem on the clients solved - they are no longer swapping. This has let us run out to a different problem - RAM demands on the data services. :-)
Good progress though, last night's run yielded 5 billion triples loaded in just under 10 hours for an average throughput of 135k triples per second. Max throughput was just above 210k triples per second. 1 billion triples was reached in an astonishing 78 minutes.
Configuration was 20 data services (10 blades with 2 data services each), 8 client services (4 blades with 2 each), and one blade for centralized services.
Note that these times are for simple RDF load - no closure. Previous tests have demonstrated that closure takes about as long as simple load - so double the time and halve the throughput to get our numbers with closure. Still quite impressive.
posted by Mike Personick at
1:22 PM
0 Comments:
Post a Comment
<< Home