SliTaz SliTaz Forum

You are not logged in.

#1 2015-12-03 22:45:40

emninger
Member
Registered: 2015-10-24
Posts: 53

Question concerning ad blocking hosts

I see in the tazpanel there is the option to install hostfiles for ad&malware blocking. I'm not familiar with that.

Normally i set my machine up in that way:

privoxy as filtering proxy > socksproxy tor (sometime i use(d) polipo whereto i add a forbidden file; following the instructions of bodhizazen.net). I prefer a forwarding proxy for the use of tor, since there i can use the socks 4a option - which is unavailable in firefox.

Now i'd like to understand, how that hostfiles work (may be also an advice which one out of the four would be preferrable smile, in case) and if they interfere with the above proxy settings. Or, alternatively, if they could be used as a forbidden file or as an external redirector in polipo?

I think, privoxy would a) not as easy to cooperate with them b) probably wouldn't need them (but, sometimes, it has it's own kind of problems especially as for finetuning to not block sites which are NOT unwanted).

Thanks a lot in advance for your patience.

Offline

#2 2015-12-04 23:59:39

lexeii
Administrator
Registered: 2012-03-21
Posts: 1,853

Re: Question concerning ad blocking hosts

Hi emninger,

I wrote the TazPanel code to maintain these lists. I use the same combined hosts list in my Android smartphone to block advertisement  in browser and in free programs with advertising load, and I think this list will be useful on the SliTaz desktop too.

Of course, it will not replaced specialized tools, but it is quite good and simple. How it works: DNS not used for hosts listed in the hosts file, system just reads their IP from file. In all cases IP is 127.0.0.1 or 0.0.0.0 also known as localhost. So, instead of downloading http://ad.example.com/banner.gif browser downloads http://localhost/banner.gif which is absent, and you'll see no advertising. You can use TazPanel to update lists, as well as add or remove subscribe, and individually add or remove list entries.

I want to say I never used Tor, Privoxy or Polipo, and can't tell about them.

Offline

#3 2015-12-05 12:55:59

emninger
Member
Registered: 2015-10-24
Posts: 53

Re: Question concerning ad blocking hosts

Thanks for all that info. Please forgive, if i'm asking a dumb question:

As i understand it the 4 adblockers are lists (files with a kind of blacklisted urls), correct? The browser reads them and directs them to a http://localhost/banner.gif. But how does the browser understand that it should read them?

Privoxy or Polipo are proxies, 127.0.0.1:8118 respectively localhost:8123 - the port can be configured. So, either you point the browser to use that proxy to have all going out http demands to be filtered and then forwarded to the parent (socks) proxy, tor, in my case; https obviously can be filtered but would be forwarded. Or you define them as systemwide proxies for http and https demands.

Privoxy is Polipo itself does not have filtering but the option to define an external forbidden file (which can be created following this instructions: http://bodhizazen.net/Tutorials/TOR#Adblock ) or an external redicrector (cfr. http://www.pps.univ-paris-diderot.fr/~jch/software/polipo/polipo.html#External-redirectors ) - or eventually both (i do not know, never tried that). If i knew how the lists in tazpanel work, may be i could use one (or more) of them as either external redirector or as forbidden file (the procedure to build one as described in the link above is nt really difficult but to have it build automatically by the system obviously would be a lot smarter wink ).

TIA

PS. I forgot to ask: Do i understand right, you are using all the 4 files, shown in the tazpanel, together?

Offline

#4 2015-12-05 14:46:30

lexeii
Administrator
Registered: 2012-03-21
Posts: 1,853

Re: Question concerning ad blocking hosts

Hi emninger,

I can't say better than Wikipedia. Please read the article https://en.wikipedia.org/wiki/Hosts_%28file%29#Extended_applications

PS - Yes, all the 4.

My PS.

Reading http://bodhizazen.net/Tutorials/TOR#Adblock . Download easylist. Maybe you can convert hosts rules to easylist rules in this manner:

hosts:

[c]. . .
0.0.0.0 005.free-counter.co.uk
0.0.0.0 006.free-counter.co.uk
. . .[/c]
easylist:

[c]. . .
@@||005.free-counter.co.uk/*
@@||006.free-counter.co.uk/*
. . .[/c]
(I not familiar with the easylist syntax, sorry.)

Hosts file can't block individual resources. It blocks whole hosts, without differences to protocol. And it works system-wide, for browser and for wget and all other networking tools:

[c]wget http://005.free-counter.co.uk/
Connecting to 005.free-counter.co.uk (0.0.0.0:80)
wget: can't open 'index.html': Permission denied

wget https://005.free-counter.co.uk/
Connecting to 005.free-counter.co.uk (0.0.0.0:443)
wget: error getting response: Connection reset by peer

wget ftp://005.free-counter.co.uk/
Connecting to 005.free-counter.co.uk (0.0.0.0:21)
wget: can't connect to remote host (0.0.0.0): Connection refused

ping -c3 005.free-counter.co.uk
PING 005.free-counter.co.uk (0.0.0.0): 56 data bytes
64 bytes from 127.0.0.1: seq=0 ttl=64 time=0.122 ms
64 bytes from 127.0.0.1: seq=1 ttl=64 time=0.148 ms
64 bytes from 127.0.0.1: seq=2 ttl=64 time=0.150 ms

--- 005.free-counter.co.uk ping statistics ---
3 packets transmitted, 3 packets received, 0% packet loss
round-trip min/avg/max = 0.122/0.140/0.150 ms[/c]
(In reality I ping my localhost)

Offline

#5 2015-12-05 16:57:33

emninger
Member
Registered: 2015-10-24
Posts: 53

Re: Question concerning ad blocking hosts

Thanks Aleksej, i think i got it big_smile I wasn't aware of this way to block advertising, could have saved a lot of time trying to configure proxies. And it's not that i'm "married" with easylist. Your hosts seem to work nicely.

Is there a way to get an automated update?

Offline

#6 2015-12-06 02:02:18

Darjeeling
Member
Registered: 2012-02-06
Posts: 210

Re: Question concerning ad blocking hosts

Since years I've been using the hosts file from mvps.org which is excellent, although with 500+ Kb quite large. I have got so much used to it that whenever I am working on other people's computer I am shocked how shit the Internet actually looks :-) But this is of course a matter of personal preference. The file is updated each 4-6 weeks. I have a script (run out of local.sh) that looks if a new version is available and if so, downloads it to /etc/hosts ...

[c]Originally I had the script posted here but I removed it after a few minutes because
I remembered too late that it had escape sequences in it. I do hope I haven't once
again messed up your RSS feed, Aleksej.[/c]
A while back the mvps maintainers have switched to from 127.0.0.1 to 0.0.0.0 because of some changes in Windows. This caused problems in SliTaz 3.0 which I'm still using, so my script changes the file back to 127.0.0.1 after the download.

Offline

#7 2015-12-06 12:53:52

lexeii
Administrator
Registered: 2012-03-21
Posts: 1,853

Re: Question concerning ad blocking hosts

Hi Darjeeling,

Oh, don't worry, nothing was broken that time.

You can validate our RSS feed using this link:

https://validator.w3.org/feed/check.cgi?url=http%3A%2F%2Fforum.slitaz.org%2Frss%2F

I use RSS to check and read new messages on our forum (as well as check updates on the hg.slitaz.org/wok , new packages on cook.slitaz.org , and many other forums, blogs and web sites) using my smartphone. That time, forum topic of interest just was not updated. I've got posts from other forum topics (incl. flood from our Portuguese user) but did not even know that people wrote to me in the interested topic :-$

This time nothing broke, I've seen only a long number without separators.

As for on-topic. I want to note, hosts file worked as expected using "0.0.0.0" in my current Rolling. IIRC I even changed "127.0.0.1" to "0.0.0.0" in one or pair of four lists to be consistent with others.

Offline

#8 2015-12-06 15:19:41

Darjeeling
Member
Registered: 2012-02-06
Posts: 210

Re: Question concerning ad blocking hosts

> This time nothing broke ...

I'm glad to hear it.

> ... I've seen only a long number without separators.

Yes, this was strange. When I saw it I removed the whole script. The long number without separators was actually a "| tr -d" followed by series of octal characters separated by backslashes, which was needed to make the date in the /etc/hosts file comparable to the date on the mvps.org web page. But I have learned from this: No more backslashes followed by strange characters in forums! :-)

Regarding RSS: I never really got the hang of it, maybe because I don't have a smartphone where, I guess, it is a much more convenient way to keep oneself up to date. I know some people who use RSS the way you use it and they swear by it. But for the time being I'm still clicking bookmarks to see what has happened while I was away from the computer ...

Regarding 5.0: I'm getting closer now to finally making the switch from 3.0 to my justx-based 5.0 diy SliTaz. The reason for the delay is that both SliTaz and SpaceFM are still changing quite a bit and, even worse, that I often don't have time to do anything for weeks. I really do like SpaceFM now because it is much less buggy than it was (0.9.4) when it served for a short while as FM in SliTaz 5.0. It is also much more like the original 0.5x series of PCManFM (no gvfs, udisks, polkit...) but more versatile and powerful. Maybe the devs should take another look at SpaceFM when SliTaz 5.0 is out of the door and development for 6.0 begins?

Offline

#9 2015-12-06 15:57:00

lexeii
Administrator
Registered: 2012-03-21
Posts: 1,853

Re: Question concerning ad blocking hosts

I've used nice technique to check if file changed on the server:

http://hg.slitaz.org/tazpanel/file/tip/hosts.cgi#l289

No need to download whole file, no need to parse date inside it.

First you determine date of your existing list, then ask server if file modified since that. Then download file if it modified.

You can use it in your script also.

Offline

#10 2015-12-06 19:19:59

Darjeeling
Member
Registered: 2012-02-06
Posts: 210

Re: Question concerning ad blocking hosts

I just had another peek at TazPanel with the iso from 22. November. It's been some time since I last tried this out and I find it amazing how much functionality you've been adding to the application. I also had a look at your hosts file code. I must admit that I understand it only partially but from what I can make out it's not all that different to what I'm doing in my script. We're both extracting the date string within the local hosts file and compare it to the date string that's buried deeply in the hosts.htm page at mvps.org. But you seem to be using the spider mode of wget (which I'm not yet familiar with) and download only the interesting bit of hosts.htm while I download the whole page once or twice per day (on each boot). But like you I only download the actual hosts file once each 4-6 weeks on or shortly after the day it has been updated.

Offline

#11 2015-12-07 18:49:10

gibor
Moderator
Registered: 2011-04-30
Posts: 1,067

Re: Question concerning ad blocking hosts

@Darjeeling

Hi Darjeeling on this post http://forum.slitaz.org/topic/question-concerning-ad-blocking-hosts#post-40525 you have suggested a good tricks. Have tried on my system, and have merged at list the site untrusted saved from noscript. The final result is at first impression good, very good.

Thank you

Offline

#12 2015-12-07 20:30:21

Darjeeling
Member
Registered: 2012-02-06
Posts: 210

Re: Question concerning ad blocking hosts

@ Gibor

... cool! It has never occurred to me to use the mvps hosts file in this way. But although I'm sure it works, using such a long list like an ad-blocker that runs inside the browser is in my opinion not the best way because ...

- it eats up more resources and slows down the browser

- it acts only on web traffic, not on the entire network traffic

- if you use multiple browsers you have to maintain multiple copies of the list

It is in my opinion much more effective and resource-efficient to filter network traffic by diverting specific IPs or URLs to localhost, which is what the /etc/hosts file does. Using the hosts file to filter undesirable or dangerous stuff is also a more universal and elegant method than browser based filtering.

But the ONE BIG advantage of a browser-based block list is that it can be easily edited within the browser and that list items can be temporarily enabled/disabled etc., which makes its use much more flexible than editing the root-protected hosts file. In the end, I guess, it's all a matter of personal preferences :-)

Offline

Registered users online in this topic: 0, guests: 1
[Bot] ClaudeBot

Board footer

Powered by FluxBB
Modified by Visman

[ Generated in 0.015 seconds, 7 queries executed - Memory usage: 1.57 MiB (Peak: 1.77 MiB) ]