SliTaz Forum
You are not logged in.
Pages: 1
While setting up a tiny mailserver with a couple of packages incl. postfix I recognized a massive growth of the resulting packed image after upgrade.
I could identify the package causing this growth, which is postfix.
During the last upgrade in (the same) Version number 2.10.2, the postfix files in /usr/sbin as well as the files in /usr/lib/postfix grew in size by a factor of 7 to 13 (from about 100-300k to between 1.3 and 1.7 M) for about 40 single files keeping the same names and structure.
I guess that this should not be by intent.
Did anybody else experience this problem?
Offline
Hi,
The only relevant recent change is this: http://hg.slitaz.org/wok/rev/c8f848a49bc1 , but I don't see why linking to a shared lib would grow an executable this much. Pascal?
Offline
Hi,
Regarding the BUILD_DEPENDS:
openssl; http://hg.slitaz.org/wok/rev/f45ab85f7718
Offline
Oh, I'm sorry if this change of mine is the culprit, and I don't understand why:
- a change of patchlevel normally doesn't change the API,
- dynamically linked objects do not include the library contents,
- so even if openssl grew, why would postfix do the same?
Offline
I mean that we must not search in postfix receipt only.
But in every package in BUILD_DEPENDS, their DEPENDS, cookutils...
Offline
Hi Filou,
Do yo know approximately when postfix was small for the last time? That would help spotting the problem.
Offline
Hi everyone,
I set up my server on 19th of december 2015. The file sizes I attached in my first post ("before update") were from that date. Upgrading to the latest build of the same postfix version leads to the second version ("after update").
The change must have taken place in the meantime.
Offline
Found this, maybe:
http://hg.slitaz.org/wok/rev/24033cc7961e
(edit: saw your post. This change was done in october, so it's not the culprit)
It turns out package db-dev provides only static libraries:
http://cook.slitaz.org/cooker.cgi?files=db-dev
shared ones are in libdb.
But if I remove libdb*a files, postfix fails to compile with:
[c]/usr/bin/ld: cannot find -ldb[/c]
although I installed shared libs. (Not sure I did this right though; I just added [c]rm /usr/lib/libdb*a[/c] in compile_rules().)
Offline
Compiling with -DNO_DB makes the programs smaller but still much bigger than expected (~500KB). So we have to search further...
Offline
I have good and bad news!
Good news: I think I found the problem, it is related to http://hg.slitaz.org/wok/rev/c8f848a49bc1, because if I compile without [c]-lpthread[/c] it results in the small binaries.
Bad news: to be able to test-compile without pthread, I have to remove [c]-ldb[/c] and DB support, because libdb depends on libpthread! So I don't know how to solve the problem.
Pascal, do you remember why you added [c]-lpthread[/c]? Because the build of postfix was broken?
Offline
Hi Lucas,
Excuse my little side note. This may be related: http://www.postfix.org/DB_README.html#pthread
Offline
Aleksej, you're welcome! What I don't understand is that the version of db has not changed, and Filou had a postfix version linked to this version, before Pascal added the link to pthread. So, postfix must have been successfully compiled with libdb and without libpthread, but how?
Offline
Lucas, I don't understand too much in the compilation. It seems like all the postfix binaries contain some static library (*.a) instead of using shared (*.so). That's all what I can suggest.
Output for the shared libraries:
[c]ldd /usr/sbin/postfix
/usr/sbin/postfix:
linux-gate.so.1 => (0xffffe000)
libldap-2.4.so.2 => /usr/lib/libldap-2.4.so.2 (0xb76a9000)
liblber-2.4.so.2 => /usr/lib/liblber-2.4.so.2 (0xb769d000)
libsasl2.so.2 => /usr/lib/libsasl2.so.2 (0xb7686000)
libssl.so.1.0.0 => /usr/lib/libssl.so.1.0.0 (0xb7618000)
libcrypto.so.1.0.0 => /usr/lib/libcrypto.so.1.0.0 (0xb7435000)
libpthread.so.0 => /lib/libpthread.so.0 (0xb741a000)
libmysqlclient.so.18 => /usr/lib/libmysqlclient.so.18 (0xb7150000)
libz.so.1 => /usr/lib/libz.so.1 (0xb713d000)
libm.so.6 => /lib/libm.so.6 (0xb7110000)
libpcre.so.1 => /usr/lib/libpcre.so.1 (0xb70d6000)
libnsl.so.1 => /lib/libnsl.so.1 (0xb70ba000)
libresolv.so.2 => /lib/libresolv.so.2 (0xb70a1000)
libc.so.6 => /lib/libc.so.6 (0xb6f2b000)
libdl.so.2 => /lib/libdl.so.2 (0xb6f26000)
/lib/ld-linux.so.2 (0xb76e8000)
librt.so.1 => /lib/librt.so.1 (0xb6f1d000)
libstdc++.so.6 => /usr/lib/libstdc++.so.6 (0xb6e55000)
libgcc_s.so.1 => /usr/lib/libgcc_s.so.1 (0xb6e3d000)[/c]
So, seems problem not in the libpthread?
Or, may it be "hybrid" case, when some part of postfix use static libpthread, while other is depends on shared libpthread? I have very weak skills, and in the most cases can only use "trial and error" method.
Offline
@llev:
postfix has been built without pthread 2 months ago... and can't build without now. I don't know why.
Offline
I used the ldd command to check again for changes. Here they are:
- libpthread.so was in the libraries before and after the update,
- libdb.so has vanished
- libpam.so was added.
Is this of any help?!?
Offline
Hi Filou,
Our [c]ldd[/c] outputs are different. Interesting why? I have installed latest postfix:
[c]tazpkg info postfix
TazPkg information
================================================================================
Package : postfix
State : installed package
Version : 2.10.2
Category : network
Short desc : fast, easy to administer, and secure mailer.
Maintainer : pascal.bellard@slitaz.org
License : other
Depends : libdb libldap pcre libssl slitaz-base-files libsasl libkrb5
libcomerr3 libmysqlclient
Build deps : db-dev openldap-dev pcre-dev openssl-dev perl mysql-dev
cyrus-sasl-dev libsasl
Web site : http://www.postfix.org/
Conf. files: /etc/postfix
Provide : sendmail
Size : 2.9M/55.9M
================================================================================[/c]
Offline
I am on cooking, hope this does not make a difference?!?
Postfix Info before update:
TazPkg information
================================================================================
Package : postfix
State : installed package
Version : 2.10.2 (new build available)
Category : network
Short desc : fast, easy to administer, and secure mailer.
Maintainer : pascal.bellard@slitaz.org
License : other
Depends : libdb libldap pcre libssl slitaz-base-files libsasl libkrb5
libcomerr3 libmysqlclient
Build deps : db-dev openldap-dev pcre-dev openssl-dev perl mysql-dev
cyrus-sasl-dev libsasl
Web site : http://www.postfix.org/
Conf. files: /etc/postfix
Provide : sendmail
Size : 1.1M/8.4M
================================================================================
and after update:
TazPkg information
================================================================================
Package : postfix
State : installed package
Version : 2.10.2
Category : network
Short desc : fast, easy to administer, and secure mailer.
Maintainer : pascal.bellard@slitaz.org
License : other
Depends : libdb libldap pcre libssl slitaz-base-files libsasl libkrb5
libcomerr3 libmysqlclient
Build deps : db-dev openldap-dev pcre-dev openssl-dev perl mysql-dev
cyrus-sasl-dev libsasl
Web site : http://www.postfix.org/
Conf. files: /etc/postfix
Provide : sendmail
Size : 2.9M/55.9M
================================================================================
looks the same, doesn't it?!
Offline
Yes Filou, it helps. It proves that static linking of libdb is the likely problem. I'm wondering: do you have the date of the "before" package you installed on the 19th of december?
Aleksej, Pascal, do you have an idea why libpam appears? Is it a dependency of another lib?
May one of you try to do properly what I tried here:
http://forum.slitaz.org/topic/excessive-size-growth-of-postfix-package#post-41402 ?
That is to say, build postfix with *.h and *.so installed, not *.a/*.la .
Offline
Filou,
Despite we are both using same package with the same version, the packages are different. This may be because package was re-builded several times without changing the version.
I compared my ldd output with yours:
[c]mkdir /tmp/check
cd /tmp/check
# download here your file "ldd_postfix_after_update.txt"
ldd /usr/sbin/postfix > my_ldd
diff /tmp/check/ldd_postfix_after_update.txt /tmp/check/my_ldd
--- /tmp/check/ldd_postfix_after_update.txt
+++ /tmp/check/my_ldd
@@ -1,22 +1,22 @@
+
/usr/sbin/postfix:
- linux-gate.so.1 => (0xffffe000)
- libldap-2.4.so.2 => /usr/lib/libldap-2.4.so.2 (0xb76e0000)
- liblber-2.4.so.2 => /usr/lib/liblber-2.4.so.2 (0xb76d4000)
- libsasl2.so.2 => /usr/lib/libsasl2.so.2 (0xb76bd000)
- libssl.so.1.0.0 => /usr/lib/libssl.so.1.0.0 (0xb764f000)
- libcrypto.so.1.0.0 => /usr/lib/libcrypto.so.1.0.0 (0xb746c000)
- libmysqlclient.so.18 => /usr/lib/libmysqlclient.so.18 (0xb7187000)
- libz.so.1 => /usr/lib/libz.so.1 (0xb7174000)
- libm.so.6 => /lib/libm.so.6 (0xb7147000)
- libpcre.so.1 => /usr/lib/libpcre.so.1 (0xb710d000)
- libnsl.so.1 => /lib/libnsl.so.1 (0xb70f1000)
- libresolv.so.2 => /lib/libresolv.so.2 (0xb70d8000)
- libc.so.6 => /lib/libc.so.6 (0xb6f62000)
- libdl.so.2 => /lib/libdl.so.2 (0xb6f5d000)
- librt.so.1 => /lib/librt.so.1 (0xb6f47000)
- libstdc++.so.6 => /usr/lib/libstdc++.so.6 (0xb6e81000)
- libgcc_s.so.1 => /usr/lib/libgcc_s.so.1 (0xb6e69000)
- /lib/ld-linux.so.2 (0xb771f000)
+ linux-gate.so.1 => (0xffffe000)
+ libldap-2.4.so.2 => /usr/lib/libldap-2.4.so.2 (0xb775a000)
+ liblber-2.4.so.2 => /usr/lib/liblber-2.4.so.2 (0xb774e000)
+ libsasl2.so.2 => /usr/lib/libsasl2.so.2 (0xb7737000)
+ libssl.so.1.0.0 => /usr/lib/libssl.so.1.0.0 (0xb76c9000)
+ libcrypto.so.1.0.0 => /usr/lib/libcrypto.so.1.0.0 (0xb74e6000)
+ libpthread.so.0 => /lib/libpthread.so.0 (0xb74cb000)
+ libmysqlclient.so.18 => /usr/lib/libmysqlclient.so.18 (0xb7201000)
+ libz.so.1 => /usr/lib/libz.so.1 (0xb71ee000)
+ libm.so.6 => /lib/libm.so.6 (0xb71c1000)
+ libpcre.so.1 => /usr/lib/libpcre.so.1 (0xb7187000)
+ libnsl.so.1 => /lib/libnsl.so.1 (0xb716b000)
+ libresolv.so.2 => /lib/libresolv.so.2 (0xb7152000)
+ libc.so.6 => /lib/libc.so.6 (0xb6fdc000)
+ libdl.so.2 => /lib/libdl.so.2 (0xb6fd7000)
+ /lib/ld-linux.so.2 (0xb7799000)
+ librt.so.1 => /lib/librt.so.1 (0xb6fce000)
+ libstdc++.so.6 => /usr/lib/libstdc++.so.6 (0xb6f06000)
+ libgcc_s.so.1 => /usr/lib/libgcc_s.so.1 (0xb6eee000)
- libpthread.so.0 => /lib/libpthread.so.0 (0xb7451000)
- libpam.so.0 => /lib/libpam.so.0 (0xb6f51000)[/c]
The significant difference that my ldd output not contains two last lines (with libpthread.so.0 and libpam.so.0). And I have no "pam" installed.
I touched the same problem some time ago that ldd returned some "extra" dependencies.
Offline
Hi Lucas,
I was spend too more time in trials and errors. No luck, I give up.
I tried to remove from receipt:
[*]"-DHAS_DB" from CCARGS
[*]"-ldb" and "-lpthread" from AUXLIBS
It fails without "-lpthread", so it restored.
Then "-ldb" magically re-appears in the postfix cooking log, that produces the same big executables, ca. 1.5 Mo size.
In the BLFS book I found option "--disable-static" in the DB package compiling.
I re-compile db, db-dev and libdb with this option, then return to postfix.
No luck, the same error as yours: "/usr/bin/ld: cannot find -ldb"
The cause of this error:
Postfix not have configure script. It begins with making of makefile by using "makedefs" script (from its sources): ftp://cook.slitaz.org/postfix/source/postfix-2.10.2/makedefs
There's defined for "Linux.3*" (I guess Linux Kernel 3.x) to include "db.h" and "-ldb" unless "-DNO_DB" option is given.
So, we have two ways:
[*]include static DB lib (with "-ldb")
[*]drop DB support completely
I think the first way is preferred.
Offline
Hi llev,
I'm afraid that because of the way I constructed my server image I can't tell the date of the "before" postfix package. Since for any changes I simply unpack my rootfs.gz, chroot and apply the changes and repack the image, the date of all files are from the last packing.
Since tazpkg info does not provide any exact build number and I always empty the package cache, I don't have any idea how I could tell the build date (I would have done already since I knew the importance...), not even in the config files or anywhere...
Should there be any idea how I could read out that date... very welcome !!!
Offline
@Aleksej:
I'm just thinking: is it possible to "go back in time" with mercurial? Then we could try to cook postfix and its dependencies at "some time ago" (assuming cookutils are not playing a role in this bug).
I'm not sure you can do this without a chroot (you don't have one, right?). I could do this on my computer if you instructed me what to do:
- how to go back in time in mercurial;
- how to cook a package and its dependencies: normally [c]cook[/c] will download dependencies (not cook them), is it sufficient to cook the dependencies first, to prevent cook from downloading them?
@Filou:
Nevermind. If the contraption I suggest to Aleksej works, dichotomy will quickly solve the problem.
(But shouldn't packing and unpacking preserve timestamps?)
Offline
Hi Lucas,
You are tell about advanced using of Mercurial, but I never use something beyond ~7 commands (st, pull -u, commit, push, add, addrem, revert). Cooker can compile specified Hg revision:
[c]tux@slitaz:~$ cooker --help
. . . . .
rev|-r Cook packages of a specific revision.[/c]
But, if I understand correctly, it will re-cook ALL the packages (too many).
Revision number you can see in the Hg web interface http://hg.slitaz.org/wok/
Here you can see some very useful tools.
Search: http://hg.slitaz.org/wok/log?rev=postfix
File log: http://hg.slitaz.org/wok/log/c8f848a49bc1/postfix/receipt
See receipt with annotations (committer and revision to the left side on each line): http://hg.slitaz.org/wok/annotate/c8f848a49bc1/postfix/receipt
Maybe these tools are enough to you to track changes?
As for «jumping to the past», really, as Pascal stated, you need to «downgrade» not only the postfix, but all of its dependencies, and all dependencies of all dependencies, back to that day X.
I'm not sure you can do this without a chroot (you don't have one, right?).
I using chroot to compile packages. I don't want to my system became garbage dump.
- how to cook a package and its dependencies: normally cook will download dependencies (not cook them), is it sufficient to cook the dependencies first, to prevent cook from downloading them?
In the chroot with installed all you need to compile packages:
[c]# cook postfix[/c]
You need to have /home/slitaz/wok/postfix/receipt in the chroot to have ability to cook postfix. Also you need "stuff" folder with stuff: in other words, copy "postfix" folder with all its content from "official" wok to your "chroot" wok.
All "build dependencies" are downloaded from the mirror. But if you compile some of them, "cook" will install yours packages instead of "official" mirrored packages. This way I tried yesterday to recompile "db", "db-dev" and "libdb" without static libs: "cook" used my packages when I re-cooked postfix. Next, when I gave up, I removed "db" and other folders from /home/slitaz/wok chroot, and removed "db" packages from /home/slitaz/packages to return to regular official "db" packages.
Offline
Pages: 1
[ Generated in 0.024 seconds, 7 queries executed - Memory usage: 1.6 MiB (Peak: 1.77 MiB) ]