naschenweng.info

In a previous post I explained how to gather statistics for your Netgear DG834 DSL modem router. Thanks for all the donations!! I have made some subtle changes in the router.cgi to display SNR/attenuation in one chart and display the latest values:

The scripts don't just run on a Synology DiskStation. I have received reports that people integrated the scripts with their Linux boxes, QNAPs and others. In some instances, people modified the scripts (the code is easy enough to modify even for a Linux newbie) to make them work with other routers.

Again, please note, that out of the box, the scripts will work with any modem router supporting telnet and "adslctl" on the router. I would love to make SNMP or HTTP-polling functionality available, but have not got any router supporting this at the moment.

If you want to use the scripts, donate below (you require a running version of Perl with Net::Telnet and Net::Ping, rrdtool -- the scripts will run on the Synology out of the box).

In a previous post I explained the significance of SNR and attenuation for ADSL lines.

I have spent now a significant amount of time to write scripts to retrieve the ADSL-statistics for my Netgear DG834v4 ADSL router. The script will equally work on any Broadcom based modem-router which allows access to telnet and supports the command "adslctl".

The solution is really simple and consists of one perl-script collating the information and a CGI-script which then produces the stats in real time from your webserver:

I have deployed the solution on my Synology DiskStation DS107+ (FW722) and had the solution running now for a day. If you want to run this on the Synology, you will need to install rrdtool via ipkg. If you want to run it on another Linux-based environment, ensure that you have access to the command-line version of rrdtool and rrdcgi (which comes bundled with rrdtool).

My current setup produces stats every 15 minutes and reports on the following stats:

• Ping statistics. Currently against my local ISP - saix.net, but scripts are easy to modify to include your own pings
• ADSL signal to noise ratio (up / down)
• ADSL attenuation (up / down)
• Power output (up / down)
• Maximum ADSL data rate (up / down)
• ADSL sync speed (up / down)

All stats are reported per hour, day, week and month. In the future I want to enhance the scripts to provide additional useful reports for the Netgear DG834.

By now you are probably asking "Where is the download-link?" --- well here is the thing, my hosting and the popularity of this site is starting to cost me a bundle. So I am introducing donation-ware. It is up to you to decide if you want to donate USD 1,00 or a million (yeah baby) using PayPal via the donation-button below. The donation is absolutely save and you will receive the download-link automatically via email (link is valid for 24 hours). Further, I will email you any updates to the scripts and will look at implementing any recommendations you have.

I think this sounds like a fair deal, especially since it took me in access of 60 hours just getting those scripts done (don't get a fright though - the scripts are simple - that's the beauty of the solution).

If you want to use the scripts, donate below (you require a running version of Perl with Net::Telnet and Net::Ping, rrdtool -- the scripts will run on the Synology out of the box).

In a previous post I described how to install Squid on the Synology. This small tutorial will now explain how to display your usage behaviour in Squid via Webalizer.

You need to have ipkg installed and need to have an understanding on how to include 3rd party apps (i.e. via phpsrc).

1. Install webalizer: ipkg install webalizer
2. Create a new config file: cp /opt/etc/webalizer.conf.sample /opt/etc/webalizer.conf
3. Adjust your log-file entry and point it to your Squid-access log: LogFile /opt/var/squid/logs/access.log
   
4. Adjust your output directory (this is the HTML-directory from which you want to display your stats): OutputDir /volume1/webapps/scripts/squid/usage
5. Create a cronjob entry (remember that the crontab needs to have tabs instead of spaces): 5 0 * * * root /opt/bin/webalizer -F squid -A 100 -C 100 -R 100 -S 100 -U 100 -e 100 -E 100 -Q

The above will create usage statistics every day (5 minutes past midnight) and publish the stats to the output-directory.

Some more cool stuff:

• Use HideUrl to hide sites from the statistics. I typically hide access to all my local servers, since I am only interested in the outbound usage.
• Use GroupUrl and HideUrl together to group certain websites and make it one line-item in the stats. I access the PS3ZA-forum frequently, so I don't want to have every URL show, since it fills up the stats. Using GroupUrl www.ps3za.co.za/forum and HideUrl www.ps3za.co.za/forum will create one line-item in the stats. You can do the same with your RSS-feeds and others.
• To avoid your logs filling up, you want to add a cronjob to rotate your squid access-logs (remember those TABS - this job will rotate the logs 30 minutes past midnight every month): 30 * 1 * * root /opt/sbin/squid -k rotate. Important: Webalizer don't rotate the logs during the month - I was not able to get Webalizer to analyse those logs and you will lose your monthly stats.

In a previous post I detailed how to install Squid on a Synology DS107+. This post explains how you are able to include Squid's cache-manager UI. This post is for advanced users and I am not going to re-explain how to configure a 3rd party application in the DS.

1. Adjust your cachemgr.conf and include your Squid's url. In my case this is an extra line with 172.16.0.97:8080
2. Ensure that you have configured the correct ACLs for the cache-manager. If you took my squid.conf then you are fine.

3. Test the cache-manager: squidclient -h 172.16.0.97 -p 8080 mgr:info

   Code:

   	Squid Object Cache: Version 2.6.STABLE21
   	Start Time:  Fri, 19 Sep 2008 13:57:53 GMT
   	Current Time:  Fri, 19 Sep 2008 14:08:44 GMT
   	Connection information for squid:
   	Number of clients accessing cache:  2
   	Number of HTTP requests received:  154
   	Number of ICP messages received:  0
   	Number of ICP messages sent:  0
   	Number of queued ICP replies:  0
   	Request failure ratio:   0.01
   	Average HTTP requests per minute since start:  14.2
   	Average ICP messages per minute since start:  0.0
   	Select loop called: 135277 times, 4.810 ms avg

4. The Squid installation comes with the cachemgr.cgi in /opt/libexec/cachemgr.cgi. Best way is to symlink (ln -s) the file into your 3rd party-directory

With the above information you will be able to tune Squid accordingly.

There you are, with that 1TB NAS and you surf mostly the same websites and in the process waste plenty of time waiting on downloads. So why not install your own Squid-proxy server on your NAS?

With the Synology and the pre-requisite of having ipkg installed - this takes no more than 10 minutes.

Update (2008-12-22): I have adjusted the Squid-configuration to block websites for unlisted IP-addresses. If you don't require this (and want your kids to download several gigs of You Tube-videos) then delete the lines acl nonblockedip, acl blocksites and http_access deny blocksites.

In my example, my NAS IP is 172.16.0.97 and my IP range on my LAN is 172.16.0.0 - adjust this accordingly below:

1. Install squid: ipkg install squid

2. Adjust Squid's config-file located in /opt/etc/squid/squid.conf:

   Code:

   	## SQUID CONFIG
   	cache_mgr Gerd@Naschenweng.info
   	## Those are the ports the proxy is going to listen to
   	http_port 172.16.0.97:3128
   	http_port 172.16.0.97:8080
   	#  TAG: visible_hostname
   	#  The host-name of the proxy-server. Can really be anything
   	visible_hostname MuffinStationProxy
   	# DISK CACHE OPTIONS
   	# -----------------------------------------------------------------------------
   	# Disk-cache options. Just adjust the cache-siz (in my case 20GB)
   	cache_replacement_policy lru
   	cache_dir ufs /opt/var/squid/cache/ 20000 16 256
   	minimum_object_size 0 KB
   	maximum_object_size 2097152 KB
   	maximum_object_size_in_memory 1024 KB
   	# MEMORY CACHE OPTIONS
   	# -----------------------------------------------------------------------------
   	#  TAG: cache_mem  (bytes)
   	cache_mem 8 MB
   	memory_replacement_policy lru
   	# ACCESS CONTROLS
   	# -----------------------------------------------------------------------------
   	acl all src 0.0.0.0/0.0.0.0
   	acl manager proto cache_object
   	acl localhost src 127.0.0.1/255.255.255.255
   	acl our_networks src 172.16.0.0/24 172.16.1.0/24
   	acl to_localhost dst 127.0.0.0/8
   	acl nonblockedip src 172.16.0.3 172.16.0.5
   	# childblocks
   	acl blocksites dstdomain "/opt/etc/squid/restricted-sites.squid"
   	#Block childblocked sites
   	http_access deny blocksites !nonblockedip all
   	http_access allow manager localhost
   	http_access allow manager our_networks
   	http_access deny manager
   	# Allow all clients from my network
   	http_access allow our_networks
   	# And finally deny all other access to this proxy
   	http_access deny all
   	#Allow ICP queries from everyone
   	icp_access allow all
   	# LOG-FILES
   	# -----------------------------------------------------------------------------
   	access_log /opt/var/squid/logs/access.log squid
   	#cache_log none
   	#cache_log /opt/var/squid/logs/cache.log
   	#cache_access_log none
   	#cache_access_log /opt/var/squid/logs/access.log
   	#cache_store_log none
   	#cache_store_log /opt/var/squid/logs/store.log
   	# OPTIONS FOR TUNING THE CACHE
   	# -----------------------------------------------------------------------------
   	#  TAG: cache
   	#  A list of ACL elements which, if matched, cause the request to
   	#  not be satisfied from the cache and the reply to not be cached.
   	#  In other words, use this to force certain objects to never be cached.
   	#
   	#  You must use the word 'DENY' to indicate the ACL names which should
   	#  NOT be cached.
   	#
   	#  Default is to allow all to be cached
   	#We recommend you to use the following two lines.
   	acl QUERY urlpath_regex cgi-bin \?
   	cache deny QUERY
   	refresh_pattern ^ftp:    1440  20%  10080
   	refresh_pattern ^gopher:  1440  0%  1440
   	refresh_pattern .           0  20%  4320
   	refresh_pattern \.gif 1440 50% 40320 reload-into-ims
   	refresh_pattern \.jpg 1440 50% 40320 reload-into-ims
   	refresh_pattern \.tif 4320 50% 43200
   	refresh_pattern \.png 1440 50% 40320 reload-into-ims
   	refresh_pattern \.jpeg 1440 50% 40320 reload-into-ims
   	refresh_pattern ^http://*.google.*/.* 720 100% 4320
   	# refresh patterns to enable caching of MS windows update
   	refresh_pattern windowsupdate\.microsoft\.com/.*\.(cab|exe|psf) 4320 100% 120960 reload-into-ims
   	refresh_pattern update\.microsoft\.com/.*\.(cab|exe|psf) 4320 100% 120960 reload-into-ims
   	refresh_pattern office\.microsoft\.com/.*\.(cab|exe|psf) 4320 100% 120960
   	refresh_pattern windowsupdate\.com/.*\.(cab|exe|psf) 4320 100% 120960 reload-into-ims
   	refresh_pattern download\.microsoft\.com/.*\.(cab|exe|psf) 4320 100% 120960 reload-into-ims
   	refresh_pattern microsoft\.com 4320 100% 10080
   	pipeline_prefetch on
   	# Apache mod_gzip and mod_deflate known to be broken so don't trust
   	# Apache to signal ETag correctly on such responses
   	acl apache rep_header Server ^Apache
   	broken_vary_encoding allow apache
   	# Leave coredumps in the first cache dir
   	coredump_dir /opt/var/squid/cache
   	# Disable cachemgr password
   	cachemgr_passwd none all

3. Take note from my above config, that I chose a cache-size of 20(!) GB (cache_dir).
4. Validate your Squid configuration with squid -k parse
5. Create the Squid cache-directories with squid -z
6. Start Squid manually to check for errors: squid -NCd1
7. Create a symbolic link so that Squid starts automatically: ln -s /opt/etc/init.d/S80squid /usr/syno/etc/rc.d/
8. Once you restart the NAS, Squid should be started automatically (log files are in /opt/var/squid/logs)

Dummy error: Happened to me - if Squid starts and you don't notice any improvements in browsing speed, make sure that you have your browser's proxy settings adjusted

IMPORTANT: As I have the caching server within a DMZ/Firewall, security-concerns are secondary. All users having access to the LAN and fall within the IP-range will automatically have access to the caching-proxy. The implementation of Squid was for improving the browsing/web-experience (speed has improved by almost 200% and average bandwidth consumption dropped by 30%).

Read more »

In a previous article I showed how to make the Synology reboot a router via a cronjob.

There are many more options - such as automatically rebooting your Synology NAS thereafter - you would just extend the script by another two lines:

Code:

Synology is really rocking and have just released the latest beta. I suggest that you sign-up before the beta closes - I did my upgrade last night and everything is working good. Amongst the enhancements (I was looking forward to the hibernation fix for my Western Digital Green-HD):

• RAID 6 Support: RAID 6 volume type cuts off the risk of volume failure and provides extra data protection.
• NFS Support: With built-in NFS option, admin can quickly enable NFS and set up privileges on the UI by a few checks only.
• Virtual Host: Hosting multiple websites on one single server for a more efficient and economical purpose; both IP-based and name-based virtual hosting are supported.
• Batch Users Creating: By importing the user list, admin can now efficiently create a number of user accounts on a batch basis.
• HDD S.M.A.R.T.: The S.M.A.R.T. information and S.M.A.R.T. test are available on the management UI for admin to monitor, detect, and report the hard disks status, in a way to alert possible HDD failures. (Not supported on DS-101j)
• Enhanced Local Backup: Enables multiple scheduled Local Backup tasks with the assistance of Local Backup wizard, the backup destination can be either the external HDD or the shared folders on the internal HDD. From now on, the external attached HDD (EXT3 or FAT format) can be used as storage and backup disk simultaneously.
• Enhanced FTP Server: The advanced privilege of "write-only" is supported. In addition, limiting the maximum connections from each individual IP provides better bandwidth management together with the bandwidth restriction option. UTF-8 filename support which better handles multilingual characters is also optional.
• New Download Station 2: Other than BitTorrent/FTP/HTTP download, new download channels eMule and NZB are now supported by the Download Station 2. In addition, partial files download from BitTorrent is also allowed. Download Station 2 is now separated from the management UI and stands on an individual webpage.
• Enhanced File Station: Admin can designate his/her own File Station with a different port as well as customize webpage banner, so that users will login the File Station independent from the NAS server management UI. The advanced privilege of "write-only" is also supported.
• Enhanced Audio Station: No USB speakers are required for playing music stored on the server anymore. The new streaming mode streams .mp3 audio files to the client PC(1) and delivers by built-in speakers. A built-in playlist "Random100" will randomly choose and play 100 songs each time it is selected. (Not supported on DS-101j)
• Enhanced Photo Station 3: Admin can now customize Photo Station 3 background and theme in his/her own fashion, and sort photos and albums according to personal preference in the Settings area. Language setting is also modifiable based on his/her choice. The media RSS feed is now supported. Moreover, users can now enjoy the Photo Station 3 with PicLens dazzling full screen 3-dimensional photo slideshow(2).
• Enhanced Windows ADS Compatibility: Better handles the Windows domains especially for those with a large number of domain users. Windows Server 2008 is also supported now. (Not supported on DS107e, DS108j, DS-106e, DS-106j, and DS-101j)
• Enhanced Mac OS Usability: Synology NAS servers will be automatically found by Bonjour protocol, including Finder and Safari browser (Bookmarks > Bonjour).
• Enhanced iTunes Server: m4a and m4p file formats are correctly indexed on iTunes server with tags information.
• Enhanced Hibernation: Improved hibernation mechanism is introduced particularly to solve the problem of Western Digital Green Power series HDDs' not sleeping while in hibernation.
• Enhanced Concurrent Connection Capability: The maximum concurrent connection number is now expanded to 256. (Available on DS508, RS408, and RS408-RP)
• Perl Support: Perl 5.8.6 is formally included in Synology Disk Station Manager 2.0. (Not supported on DS-101j)