stress help Help me prove macid can scale! (Macid stress tests results for happs-tutorial toy job board disappointing so far.) - FireBoard

	FireBoard	Forum Threaded View Rules Help

Welcome, Guest
Please Login or Register. Lost Password?

FireBoard

Health

Psyhical health

stress help Help me prove macid can scale! (Macid stress tests results for happs-tutorial toy job board disappointing so far.) (1 viewing) (1) Guests

Favoured: 0

TOPIC: stress help Help me prove macid can scale! (Macid stress tests results for happs-tutorial toy job board disappointing so far.)

Forum Tools

#1163

parnell (Visitor)

stress help Help me prove macid can scale! (Macid stress tests results for happs-tutorial toy job board disappointing so far.)

but I think he was asking lemmih. Parnell, why do you think there is a space _link_ in the new version of job board state? My thinking was that 200,000 records in ram is simply all my hardware can take. Is this naive? On Oct 15, 7:43 am, tphyahoo < This e-mail address is being protected from spam bots, you need JavaScript enabled to view it wrote: I did pretty much what Lemmih suggested, and it's in darcs. Switched from set to map, and put more action specific :: Update actions and... well, it was a lot of little changes, but that's the gist of it. On Oct 14, 11:34 pm, parnell < This e-mail address is being protected from spam bots, you need JavaScript enabled to view it wrote: Could you post what you did to fix the space leak? Cheers, Parnell On Oct 14, 2:53 pm, Lemmih < This e-mail address is being protected from spam bots, you need JavaScript enabled to view it wrote: On Mon, Oct 13, 2008 at 5:18 PM, tphyahoo < This e-mail address is being protected from spam bots, you need JavaScript enabled to view it wrote: That's only ~130 transactions per second. Are you maxing out your CPU? Okay, I made things a bit more precise. The web frontend to the stresstest now gives me timing feedback: Stress Test Completed Users 642 to 841 have been inserted. Stress test time: 202 secs And the checkpoint + shutdown gives me timing feedback ..... added useruser840 added useruser841 C-c C-ccreating checkpoint time: Mon Oct 13 17:07:50 CEST 2008 shutting down system time: Mon Oct 13 17:10:41 CEST 2008 shutting down system, time: time: Mon Oct 13 17:10:41 CEST 2008 So both the webform stresstest and the shutdown take about 3 minutes. 40000 transactions / 180 seconds = 222 transactions per seconds, which is in the same ballpark as you say. I'm not sure if I'm maxing out my cpu, what can I do to tell? Are you seeing dramatically faster times? I am. By fixing a space leak and tuning the GC options, these are the times I'm getting: Users 801 to 1000 have been inserted. Stress test time: 8 secs added user: user998 added user: user999 added user: user1000 creating checkpoint time: Tue Oct 14 21:23:13 CEST 2008 shutting down system time: Tue Oct 14 21:23:16 CEST 2008 shutting down system, time: time: Tue Oct 14 21:23:16 CEST 2008 My hardware is slightly newer: david@desktop:~$ uname -a Linux lemmih-desktop 2.6.24-16-generic #1 SMP Thu Apr 10 12:47:45 UTC 2008 x86_64 GNU/Linux david@desktop:~$ cat /proc/cpuinfo processor : 0 vendor_id : AuthenticAMD cpu family : 15 model : 75 model name : AMD Athlon(tm) 64 X2 Dual Core Processor 4600+ stepping : 2 cpu MHz : 1000.000 cache size : 512 KB [snip] It is kind of a junky old laptop I'm working on. thartman@thartman-laptop:~/happs-tutorialuname -a Linux thartman-laptop 2.6.24-19-generic #1 SMP Wed Aug 20 22:56:21 UTC 2008 i686 GNU/Linux thartman@thartman-laptop:~/happs-tutorialcat /proc/cpuinfo processor : 0 vendor_id : GenuineIntel cpu family : 15 model : 2 model name : Mobile Intel(R) Celeron(R) CPU 2.00GHz stepping : 9 cpu MHz : 1995.006 cache size : 256 KB fdiv_bug : no hlt_bug : no f00f_bug : no coma_bug : no fpu : yes fpu_exception : yes cpuid level : 2 wp : yes flags : fpu vme de pse tsc msr pae mce cx8 sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe up pebs bts sync_rdtsc cid xtpr bogomips : 3994.41 clflush size : 64 Make sure you don't get bitten by laziness and try +RTS -c -A5m -RTS.

Report to moderator Logged

Logged

The administrator has disabled public write access.

#1164

tphyahoo (Visitor)

stress help Help me prove macid can scale! (Macid stress tests results for happs-tutorial toy job board disappointing so far.)

How much memory usage are using seeing? After inserting 200k jobs, HAppS on my box is using 245megs. We could probably get that number even further down by using a more compact representation of strings. I've done a bit more work (up on darcs). Interestingly I'm seeing much better performance on my linode than on my development laptop, though both machines have similar specs. On my linode, I've managed to insert 25,000 users (*200 jobs/user), which is at least the same order of magnitude as 100,000, my goal. I find I can insert 1000 users at a time, and thusly work my way up to higher user counts. However if I try to insert more than 1000 at a time the uml hypervisor kills my happs process. (linode server has 360M of ram and 256M of swap.) Maybe it's some quirk of virtualized os? Or maybe just because it's saving ram not on not running xwindows and so on? Anyway, there are now 3 stress tests:

Report to moderator Logged

Logged

The administrator has disabled public write access.

#1165

Lemmih (Visitor)

stress help Help me prove macid can scale! (Macid stress tests results for happs-tutorial toy job board disappointing so far.)

Report to moderator Logged

Logged

The administrator has disabled public write access.

#1166

Lemmih (Visitor)

stress help Help me prove macid can scale! (Macid stress tests results for happs-tutorial toy job board disappointing so far.)
Anyway, there are now 3 stress tests:

Report to moderator Logged

Logged

The administrator has disabled public write access.

#1167

tphyahoo (Visitor)

stress help Help me prove macid can scale! (Macid stress tests results for happs-tutorial toy job board disappointing so far.)
On Wed, Oct 15, 2008 at 1:52 PM, tphyahoo < This e-mail address is being protected from spam bots, you need JavaScript enabled to view it wrote: Anyway, there are now 3 stress tests:

Report to moderator Logged

Logged

The administrator has disabled public write access.

#1168

Lemmih (Visitor)

stress help Help me prove macid can scale! (Macid stress tests results for happs-tutorial toy job board disappointing so far.)
If so, are there any performance tradeoffs involved? There are. CompactMap's are quite a bit slower than ordinary maps. Try to void newtyped keys. The newtypes unfortunately keep dictionaries around which hurt performance. Here are some benchmarks: http://darcs.haskell.org/~lemmih/compact-map/benchmarks/