SliTaz Forum
You are not logged in.
Hi Everyone,
This is my first post so I want to say thank you all for such an awesome distribution! It is fast, compact and customizable -- exactly what most linux hackers need. I have found some performance issues that seem to be related to process scheduling for both the redis-benchmark and some internal python testing runs. I get somewhat sporadic performance on both the redis benchmark (generally giving 90Kops/s, 60Kops/s or 30Kops/s depending on where the client and server processes land) and also internal tests (running in 5s, 3s or 1.5s sporadically).
With my internal python testing I went and compiled the latest python and numpy versions from scratch, but without improvement. With the redis benchmark I was able to fix cpu locations using taskset to get consistent 90Kops/s performance. I think this is a scheduling issue. I know this isn't ubuntu (yay!), but the same tests perform consistently well with a minimal ubuntu install (500+MB, ick!). Comparing the kernel compile options between the two and browsing through the kernel code I think it is one of the following two options (maybe both) that are not set:
CONFIG_RT_GROUP_SCHED : This is real-time group scheduling. There are several segments of the process scheduler that depend on this being set. Which includes making code run by root/kernel have rt scheduling requirements and reserving processing time for root/kernel. Having these specifications could allow more intelligent scheduling. This should probably be on for server applications and it doesn't hurt desktop performance either (99.7% free cpu on lubuntu and slitaz). The defaults leave the vast majority of processing available and it seems that the RT needs for audio/video would be improved for the desktop and reserved processing time for the system would improve servers.
CONFIG_SCHED_SMT : This is the option to enable hyperthreading. I'm not sure how this differs between the new AMD "cores" and regular Intel HT -- whether just leaving this on would affect compatibility with older CPUs or give a performance hit on non Intel CPUs. I think the scheduler identifies whether or not it should be used or it doesn't have any noticeable negative performance impact, just like SMP for old cpus without multi-core. Obviously if it breaks compatibility with old systems its a no-go, but I don't think that is the case.
When I tried to recompile my kernel there were a whole bunch of options that I didn't have a clue about and I ended up with a kernel panic. So I was wondering if anyone else has any little nagging benchmark problems? Anyone recompiled their kernel with everything the same except these two options turned on? Personally, I think these things should be turned on in the standard kernel, what do you think?
--David
Offline
UPDATE:
I had the opportunity to test the exact same redis benchmark on a dual core AMD machine and got consistent good results (yay tazusb!). So it seems like the difference is the CONFIG_SCHED_SMT option... I get the same performance on the dual core AMD machine using Ubuntu 12.04 with the generic 3.2 kernel even though the compile options include CONFIG_SCHED_SMT. So it appears that, like SMP for multi-core, CONFIG_SCHED_SMT does not cause significant overhead on non multi-threaded systems. Although I am not sure what the effect would be on the newest bulldozer/piledriver AMD chips (with their flavor of pseudo-core hyperthreading) -- I think the general consensus was linux distributions didn't have the same kernel problems that Windows 7 did when the bulldozer architecture was released. Does anyone else have experience with the CONFIG_SCHED_SMT kernel option? From the results that I got (on both AMD dual core and old hyper-threaded i3) it looks like this option should be turned on. CONFIG_RT_GROUP_SCHED was off with good results on the AMD so I'm not sure about that.
Offline
UPDATE: Compiled with CONFIG_SCHED_SMT enabled... No difference. Recompiling with CONFIG_RT_GROUP_SCHED.
Offline
Hi David,
when you are already working on compiling a new kernel and you have new boards, can you also activate pae?
I think, its just
config_highmem enabled
Offline
Yeah, kultex, I can do that (will need it myself soon). The ram limit without that set is 4GB, right? Do you know anything about the HIGHMEM4G and HIGHMEM64G options? It looks like the current cooking default is CONFIG_HIGHMEM4G (CONFIG_NOHIGHMEM off; CONFIG_HIGHMEM=y). So I think HIGHMEM64G=y (and 4G off) will allow PAE to map up to 64GB (or less), but that won't work for non-PAE cpus. So maybe a slitaz-linux-modern would be helpful as a kernel package option? That gives PAE-64GB, scheduling to handle HT or non-HT (maybe this should be default) and some other (say <5 yrs, but stable performance improving) options.
One more thing -- I'm trying to move to an in memory compile process to speed things up, but it keeps crashing on me. Directly on hardware on one machine and from a VM on another machine (i3 and athlon cpus, respectively). The crashes instantly halts both machines -- completely shut off, no warning. So going from my VM that's stable with a .vdi HD -- it's a sloooow process. Has anyone had problems compiling the kernel from a live CD (all in ram)?
Is there a linux-slitaz (slitaz kernel) maintainer? Are these posts in the right area?
--David
Offline
its the right place
I think, slitaz will never have pae by default - so also cooking has no pae kernel -it would not boot on my Portege R100
I never did a SliTaz Kernel, but I did some debian kernels, but cannot say anything until now with a new slitaz kernel
just compiles some single kernel modules, which I needed and this worked like a charm
thats the siltaz kernel - mostly done I think by Pankso
http://hg.slitaz.org/wok-stable/file/50647c0be30b/linux/stuff/linux-2.6.37-slitaz.config
I am a bit astonished to see that CONFIG_HIGHMEM=y
so have to digg in deaper
Offline
Yeah, I definitely agree that slitaz shouldn't have PAE on the default slitaz-linux package (it wouldn't boot on half of the systems I use). The CONFIG_HIGHMEM4G=y is the hack for 4G instead of up to 3G (CONFIG_NOHIGHMEM=y is like the WinXP/etc setting). CONFIG_HIGHMEM64G=y looks to be the true PAE from this reference:
http://oreilly.com/linux/excerpts/9780596100797/kernel-configuration-option-reference.html
Would a linux-slitaz-modern.tazpkg be helpful? That has kernel options/updates that are stable but only supported on CPUs less than ~4 years old be useful? There is a serious performance issue with the cooking kernel on my i3 that I'm trying to solve. If the solution isn't backward compatible to 386/486 maybe a linux-slitaz-modern would be helpful so that people with more recent processors can run "tazpkg get-install linux-slitaz-modern" to get the most out of newer configurations?
Pankso is one of the big bosses (the big boss?) right?
--David
Offline
yes, but there is no option in the config for HIGHMEM64G, that waht makes me a little bit worrying
There are more interested in a PAE kernel, I think the way how it could work - we have to make a iso with pae kernel to test, that we share throug dropbox ore whatever.
When whe have one, that is good working, we ask pankso or pascal to set it up in wok and create a slitaz-pae.iso
for cooking there exists already a 64bit kernel,
http://hg.slitaz.org/wok/file/8b9b79fb08c1/linux/stuff/linux-slitaz.config64
I think, that you have just to install all the linux64 stuff, you need
http://pkgs.slitaz.org/search.sh?package=64
Thomas
Offline
I tried to do a pae kernel a while ago
Though i did not investigate what was not working (it booted and i did some things but some applications were not working as expected).
I used the default configuration tool (i think it is accessible trough the cnfigure shell) and going through the menu what to change was pretty straightforward, if I am not mistaking (but that is already a while ago).
What I did not investigate is if i broke something else.
In another words, i think it configured on the basis of some default file which may not be the defaut slitaz config file.
Seems to me the safe side would be to use the default slitaz config file, which is availabe from any iso, copy it as the default in the configuration directory, then use the configuration tool to change whater needs to be changed rather than directly editing the file (i suspect some changes in the menu will in fact affect several variables/directives of the config file).
I have no experience in compiling a kernel or anything whatsoever (well, nothing younger than 15 years ago),this is just my 2 cents on the short try i gave.
Sorry i cannot spend more time on this now than this few comments....
Offline
Kultex & Christophe: From how both are you are pushing for PAE - to the point of hijacking threads and getting other people to do the work for you, I think both of you have gone a bit overboard.
So let me say this: You cannot magically make a program use PAE that wasn't designed to use it. In fact there is no magic compile option that suddenly super boosts Linux systems and programs like you guys try to find every couple of months. It doesn't exist. What does exist is a combo of compile options when used the right way can give the proper boost - but none that work alone. This is also why I think David isn't noticing any improvements by enabling one or the other option in the kernel compile. Maybe it needs both to work effectively.
Offline
I did get a significant improvement! With the CONFIG_RT_GROUP_SCHED option enabled (also had the CONFIG_SCHED_SMT option on, but that individually did not help). I just got ~20% improvement on the redis benchmark and my python benchmark. This was on athlon and i3 architectures. I am still having a scheduling issue on the i3 so that I get a 100%, 50% and 25% performance depending on where the processes land. Although with the CONFIG_RT_GROUP_SCHED turned on I saw 20% improvement on all three core scheduling possibilities (110%, 55%, 27.5%).
I agree PAE will not improve performance. It just allows you to use more than 4G of ram with a 32bit compiled kernel -- but requires a recent processor that supports it. I couldn't really test it out. I only have a 2G and 4G machine. Nothing with >4G.
I'm going do some more looking at the config options for scheduling to see if I can get the same consistency as the ubuntu kernel, but I'm running out of options. It maybe something that improved with the 3.6 kernel.
--David
Offline
trixar nobody is pushing anybody - it was just a question!
I will look for it tomorrow, and I think it is allowed to discusssomething before do something and I know, that programs are working - both with normal and pae kernel - avlinux is the example - you can try it out
and I think I will try it out tomorry and will see, if I will have the same troubles like David
@David I hope you did not have the feeling of being abused
Offline
@kultex no worries. I'm going to get to my kernel optimizations before yours anyway! 
I will be interested in a PAE because my next rig will have 8G or 16G.
Offline
I'll admit that our kernel needs work. Maybe this will help: http://kernelnewbies.org/
Offline
Update: well, that 20% performance improvement was me going off memory and accidentally comparing numbers vs VM. I am recording the results of every attempt now (along with the /proc/config.gz). I did get about a 50% improvement in HT scheduling for python and consistently high throughput on redis-benchmark on an i3. Also had no degradation on the athlon X2. That was with CONFIG_RT_GROUP_SCHED and CONFIG_SCHED_SMT enabled. I am trying a slew of options to further improve HT scheduling (but keep i386 support and non-HT performance).
Offline
sounds very promising
I had a bad day yesterday, because I all the time got errors on making the preperation, but now I think, I have got it
you cannot compile from frugal or the CD, because in frugal and CD booting /boot is missing
I use Slitaz only frugal and had all the time the errors like no rules to compile....
Maybe it is clear for evrybody, but was not for me and I will add this to the tutorial....
Offline
but it also does not work in the installed version
I all the time get the error when running:
root@slitaz:/home/tux# /usr/bin/get-linux-source
Connecting to www.kernel.org (149.20.20.133:80)
Connecting to www.kernel.org (149.20.4.69:80)
linux-3.2.tar.xz 100% |*******************************| 63540k 0:00:00 ETA
tar: invalid magic
tar: short read
/usr/bin/get-linux-source: line 44: can't open slitaz/patches: no such file
cp: can't stat 'slitaz/config': No such file or directory
ln: /lib/modules/3.2.14-slitaz/build: No such file or directory
then when I run:
root@slitaz:/usr/src/linux-3.2.14# make oldconfig && make prepare
make: *** No rule to make target `oldconfig'. Stop.
whats wrong in line 44 get-linux-source
Offline
Don't use make oldconfig
Offline
Trixar, it does not make any difference, if I use make oldconfig or make ore make install, I always get this error. I just have compiled a debian kernel, without any problem - so it seams, I am to stupid for SliTaz.
I have no time the next days, so for me postponed
@David
her is my config of a rt-pae-kernel - mainly based on AV-Linux, but with some little changes - you can try if you want - I hope I can add it as attachment
you should have installed :
# tazpkg get-install linux-cpufreq
# tazpkg get-install linux-acpi,
# tazpkg -gi cpufrequtils-008
cpufreq_ondemand is default in this kernel, but can be easyly overruled, when you add
cpufreq-set -g performance
to /etc/init.d/local.sh
not possible - you can download here: http://www.filedropper.com/config-rt-pae
Thomas
Offline
@ trixar
Not pushing anyone. Read my post more carefully.
In any case, the purpose would not be for a single process to address >4G. The kernel will transparently map the process' 4GB memory space to the >4G system (virtual and physical) mémory address space.
Or am I missing anyghing here ?
Offline
@christophe: a little late to start anything, but you guys really do carry on with a thing. My opinion on the matter still stands and there is very little that will change my mind. Not that I don't value people that disagree with me, which is why I made kultex a mod. Personally, I hate yesmen.
Offline
Noone is trying to have you change on that. Your opinion is an opinion, not a fact, and I do not think forum's rules say everyone should be of the same opinion.
Nothing personal, and there must be some compelling reason for people as unsuccessful as Apple or Canonical engineers (as well as a large chunk of others) to believe this is a viable solution to solve some issues, and I think (opinion again) you miss the point here, though I absolutely understand your position (since I love cameras that are small enough to fit in my pocket or large enough to have a large aperture zoom, and in real life, I end up with a tradeoff inbetween - there is a reason there is a market for the whole range). We have different experiences and hence different perspectives and expectations.
Can we agree this is a final word to what has, unintendly, turned to a metadiscussion ?
This being said, I will have a look at that again, if for no reason, for the sake of the fun (isn't this what slitaz also is about).
When I tried the compile was successful and Conky was showing 8GB. Had no time to investigate other issues, for which you might have a sound idea.... 
Offline
Use htop and investigate programs that use large amount of memory like Web browsers running multi-flash sites, OpenGL Games running at max detail and Virtualization software running something like Vista. That should max out your RAM pretty fast. Then you can tell us how well it holds out beyond 4Gb and if it can even go that high with only the kernel mod or if the software hard limits it instead.
Offline
fwiw i had enough time today to give it another try - it worked like a charm ....
It needs more testing, I'll let you know.
Offline
Good to hear. That's for the PAE extensions? The only options I have tried so far are the CONFIG_SCHED_SMT and CONFIG_RT_GROUP_SCHED. Neither of them decreased performance on an AMD Athlon or i3 Mobile chip, and they gave some improvement on the i3. I am putting together a new rig in the next week with 8G ram so I am definitely interested in PAE extensions. Now that my compile time should be less than 20 minutes I am going to try some more scheduling options (and compiling for generic 586+ architecture). I personally would love to see these go into a linux-pae tazpkg so that we can have the option to take advantage of newer hardware. Trixar, is there a tutorial on how to get packages into an undigest? Does that consist of making your own repo and asking people to help test, or is there an "official" undigest that hosts community package contributions?
Offline
[ Generated in 0.018 seconds, 7 queries executed - Memory usage: 1.6 MiB (Peak: 1.77 MiB) ]