This Version Of ‘Stand By Me’ Is The Best Performed One Ever! A Must See!

Displaying Upload Progress With nginx On Debian Wheezy

Version 1.0
Author: Falko Timme (timmehosting.de)
Follow me on Twitter
Last edited 02/04/2014

This tutorial shows how to use the nginx upload progress module to upload one or multiple files and display an upload progress bar for the user. This is useful, for example, if users upload large files, so that they know that something is going on in the background.

I do not issue any guarantee that this will work for you!

The nginx upload progress module isn't enabled by default. You can run

nginx -V

to check if it is enabled - you should then seen something like --add-module=/tmp/buildd/nginx-1.4.4/debian/modules/nginx-upload-progress in the output. If it's not enabled, install the nginx-extras package:

apt-get install nginx-extras

Afterwards, run

nginx -V

again - the upload progress module should now appear in the output:

root@server1:/var/www# nginx -V
nginx version: nginx/1.4.4
TLS SNI support enabled
configure arguments: --prefix=/usr/share/nginx --conf-path=/etc/nginx/nginx.conf --error-log-path=/var/log/nginx/error.log --http-client-body-temp-path=/var/lib/nginx/body --http-fastcgi-temp-path=/var/lib/nginx/fastcgi --http-log-path=/var/log/nginx/access.log --http-proxy-temp-path=/var/lib/nginx/proxy --http-scgi-temp-path=/var/lib/nginx/scgi --http-uwsgi-temp-path=/var/lib/nginx/uwsgi --lock-path=/var/lock/nginx.lock --pid-path=/run/nginx.pid --with-pcre-jit --with-debug --with-http_addition_module --with-http_dav_module --with-http_flv_module --with-http_geoip_module --with-http_gzip_static_module --with-http_image_filter_module --with-http_mp4_module --with-http_perl_module --with-http_random_index_module --with-http_realip_module --with-http_secure_link_module --with-http_spdy_module --with-http_stub_status_module --with-http_ssl_module --with-http_sub_module --with-http_xslt_module --with-ipv6 --with-mail --with-mail_ssl_module --add-module=/tmp/buildd/nginx-1.4.4/debian/modules/headers-more-nginx-module --add-module=/tmp/buildd/nginx-1.4.4/debian/modules/nginx-auth-pam --add-module=/tmp/buildd/nginx-1.4.4/debian/modules/nginx-cache-purge --add-module=/tmp/buildd/nginx-1.4.4/debian/modules/nginx-dav-ext-module --add-module=/tmp/buildd/nginx-1.4.4/debian/modules/nginx-development-kit --add-module=/tmp/buildd/nginx-1.4.4/debian/modules/nginx-echo --add-module=/tmp/buildd/nginx-1.4.4/debian/modules/ngx-fancyindex --add-module=/tmp/buildd/nginx-1.4.4/debian/modules/nginx-http-push --add-module=/tmp/buildd/nginx-1.4.4/debian/modules/nginx-lua --add-module=/tmp/buildd/nginx-1.4.4/debian/modules/nginx-upload-progress --add-module=/tmp/buildd/nginx-1.4.4/debian/modules/nginx-upstream-fair --add-module=/tmp/buildd/nginx-1.4.4/debian/modules/ngx_http_substitutions_filter_module
root@server1:/var/www#

To activate the module, we need to create a small zone in nginx.conf (in the http {} container) that is used to track uploads (I name the zone uploads in this example):

vi /etc/nginx/nginx.conf

[...]http {[...] # reserve 1MB under the name 'uploads' to track uploads upload_progress uploads 1m;[...]}[...]

Reload nginx afterwards:

/etc/init.d/nginx reload

Now open your vhost configuration file and put the following inside the server {} container (if you use ISPConfig, put this in the nginx Directives field of the website):

[...]client_max_body_size 100M;location = /upload.php { try_files $uri =404; include /etc/nginx/fastcgi_params; fastcgi_pass unix:/var/lib/php5-fpm/web1.sock; fastcgi_index index.php; fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name; fastcgi_intercept_errors on; ## Track uploads for this location on the zone defined ## above with a 60 seconds timeout. track_uploads uploads 60s;}location ^~ /progress { upload_progress_json_output; report_uploads uploads;}[...]

In my case the file that processes the uploads (i.e., the file in the action attribute of the HTML upload form) is called upload.php and resides in the document root. If your file is named differently, adjust location = /upload.php { accordingly. Also make sure you use the correct fastcgi_pass options for your website (you will probably use another socket than the one I'm using here, or you are using TCP connections instead of sockets).

In the track_uploads line it is important that you use the name of the upload_progress zone from chapter 1 (uploads in this case).

The location ^~ /progress {} container has to be taken literally, which means don't change it (well, you should change the report_uploads line if your upload_progress zone isn't named uploads). This location will be used to poll nginx for the status of the file uploads.

Reload nginx afterwards (unless you use ISPConfig which does the reload for you):

/etc/init.d/nginx reload

Next we need an upload form with some JavaScript that polls the progress location during uploads. My file looks like this - name it index.html, upload.html, etc. and put it in your document root:

'; print '- Max. Uploadsize: ' . ini_get('upload_max_filesize'); ?>

0%

Next put the following upload.php in your document root:

"; }}?>

upload.php stores uploaded files in a directory called cache, so don't forget to create that directory in your document root and give it the proper permissions so that PHP can write to it.

That's it. You can now call the file with the upload form in your browser and try to upload some files:

Falko Timme is the owner of Timme Hosting (ultra-fast nginx web hosting). He is the lead maintainer of HowtoForge (since 2005) and one of the core developers of ISPConfig (since 2000). He has also contributed to the O'Reilly book "Linux System Administration".

READ MORE HERE

Tutorial: Installing Django on Shared Hosting Service Such as Bluehost.com

Posted by Derek@TheDailyLinux in Tutorials and Guides » 5 Comments » 

This guide will walk you through how to install and run Django on a shared host such as Bluehost. Because I use Bluehost, I was able to verify the steps you see below, but you might need to modify some of the steps to work with your specific host. The whole thing should take less than 10 minutes. If I’ve done my job right, you should be able to copy/paste multiple lines to run all commands in a block as if it were a script. If you feel more comfortable, you can run each command line by line. Let’s get to it…

This step is for those of you who have an older installation of Python on your webhost. In the case of Bluehost (as of 12/6/2012), the version that is installed is 2.6.6. If you have version 2.7.0 and up (but not including version 3.0), then you can skip this step. Otherwise, follow along (you should be able to copy/paste this entire block into the terminal to run it all at once):

cd ~mkdir python27wget http://www.python.org/ftp/python/2.7.3/Python-2.7.3.tgztar xf Python-2.7.3.tgzcd Python-2.7.3/./configure -prefix=$HOME/python27/ --enable-unicode=ucs4make && make installmv ~/python27/bin/python ~/python27/bin/python27echo "PATH=\$PATH:\$HOME/python27/bin" >> ~/.bashrcecho "export LC_ALL=en_US.UTF-8" >> ~/.bashrcecho "export LANG=en_US.UTF-8" >> ~/.bashrc. ~/.bashrc

At this point, you should be able to run which python27 successfully like this:

# which python27/home1/user/python27/bin/python27

Now that Python 2.7 has been installed, we’ll need to install setuptools and pip (you should be able to copy/paste this entire block into the terminal to run it all at once):

wget http://pypi.python.org/packages/source/s/setuptools/setuptools-0.6c11.tar.gztar xf setuptools-0.6c11.tar.gzcd setuptools-0.6c11python27 setup.py installcd ~wget http://pypi.python.org/packages/source/p/pip/pip-1.1.tar.gztar xf pip-1.1.tar.gzcd pip-1.1python27 setup.py install

At this point, you should be able to run which pip successfully like this:

# which pip/home1/user/python27/bin/pip

We’re going to install MySQL-python, flup, and Django using pip (Note: If you plan on using PosgreSQL, you’ll need to install psycopg2):

pip install Django MySQL-python flup #psycopg2

At this point, you should be able to run which django-admin.py successfully like this:

# which django-admin.py/home1/user/python27/bin/django-admin.py

I’m going to name the project ‘myproject’. When you go to create your own, you’ll want to replace anything called ‘myproject’ with the project name of your choice (you should be able to copy/paste this entire block into the terminal to run it all at once):

mkdir ~/public_html/myprojectcd ~/public_html/myprojectcat > myproject.fcgi << EOF#!$HOME/python27/bin/python27import sys, osproject_name = "myproject"# Add a custom Python path.sys.path.insert(0, os.path.expanduser("~") + "/python27")sys.path.insert(13, os.getcwd() + "/" + project_name)os.environ['DJANGO_SETTINGS_MODULE'] = project_name + '.settings'from django.core.servers.fastcgi import runfastcgirunfastcgi(method="threaded", daemonize="false")EOFcat > .htaccess << EOFAddHandler fcgid-script .fcgiRewriteEngine OnRewriteCond %{REQUEST_FILENAME} !-fRewriteRule ^(.*)$ myproject.fcgi/$1 [QSA,L]EOFchmod 0755 myproject.fcgi

Lastly, we’re going to use django-admin to start our project called ‘myproject’ making sure to be in the correct directory first:

cd ~/public_html/myproject/django-admin.py startproject myproject

Now, visit your website at http://mydomain/myproject and you should see the Django start page! Have fun with it.

A final note: If you want to use the admin pages, you’ll need to follow a few more steps to get the page to show up properly with css, js, and images (you’ll need to replace mydomainname with your actual domain name).

ln -s $HOME/python27/lib/python2.7/site-packages/django/contrib/admin/static $HOME/public_html/myproject/staticsed -i "s/^STATIC_ROOT = ''/STATIC_ROOT = 'admin'/g" $HOME/public_html/myproject/myproject/myproject/settings.pysed -i "s/^STATIC_URL = ''/STATIC_URL = 'http:\/\/mydomainname.com\/myproject\/static'/g" $HOME/public_html/myproject/myproject/myproject/settings.py

Sources: Simply Argh Blog

READ MORE HERE

Kermit lives!

Posted by Jim@TheDailyLinux in General » 1 Comment » 

As a veteran of the IT industry I’ve seen software and OSes come and go, but there are some pieces of software that I seem to have used on a lot of platforms for a lot of years. One of the oldest of these is the file transfer software Kermit, from Columbia University – http://www.columbia.edu/kermit/

I’ve used kermit in the early 80's, on various CP/M systems, the very first IBM PCs and on the BBC Micro, and have been using it since on various early Unices, and other OSes that are now only memories in old fogy minds like mine. I even used hacked versions of kermit for building an email system between various computers joined by rs232 links, and then gatewayed out to the big wide world in the late 1980's.

I used it extensively in the 90's for automating and controlling transfers and connections over dial-up modems. It is still my console of choice when I need to hook up microcontrollers and single board computers with rs232. It’s configurability, features and programmability make it second to none.

However it has always erked me that it’s licensing prevented it being available in the standard repositories of the major Linux distributions. I have got used to downloading the source tar ball and compiling my own executables whenever I’ve needed kermit, and cursed silently that yum or apt-get would not simply do the job for me.

So imagine my joy when I belatedly discovered that Columbia University have cancelled the Kermit project and allowed it to be re-licensed and development continue at http://www.kermitproject.org/. The new license is a Revised 3-Clause BSD License which will at last allow Kermit to join the Free Software Family as a full member – and about time too!

If you are not familiar with Kermit, and need to go beyond where minicom etc can take you, then do check it out. It might appear a bit old school but it is very, very powerfull.

Jim

READ MORE HERE

Use CDPATH to Quickly Change Directories

Posted by Derek@TheDailyLinux in Scripting, Tips and Tricks » Add Comment » 

You can create a shortcut to frequently accessed directories by adding them to the CDPATH environment variable.  So, say I frequently access /var/www/html/.  Instead of typing cd /var/www/html, I can add /var/www/ to CDPATH and then I only have to type cd html.

Open ~/.bashrc (or ~/.bash_profile) and add the following line with your frequently used directories separated with a colon (similar to PATH variable).

export CDPATH=$CDPATH:/var/www/

Here’s an example usage:

dhildreth@hostname:~> export CDPATH=$CDPATH:/var/www/dhildreth@hostname:~> cd html/var/www/htmldhildreth@hostname:html>

There’s one caveat to using this that I’ve ran into in the past: if you are working with Makefiles and building c/c++ apps, this can potentially confuse the Makefile script. So, if you suddenly can’t build your project after adding this variable, try removing it.

READ MORE HERE

msmtp – a (fairly) simple mail submission program

Posted by Jim@TheDailyLinux in Software, Tips and Tricks » Add Comment » 

As an oldtime Unix guy, I’ve always been used to having the BSD mail utility to hand, and a suitably configured mail system, so that I can script jobs to run and email the results back to me. I use mail as a sort of glorified syslog facility. With smaller single board Linux computers we don’t always want to install a full mail setup – resources often tend to be limited. A few years back I discovered msmtp http://msmtp.sourceforge.net/.

This utility is an smtp client that submits a file in standard mail format to a mail server. It can submit plain text email or use TLS/SSL etc. I use a couple of script wrappers to emulate, sort of, sendmail and the sending part of the BSD mail utility.

Of course to use msmtp you need a mail server to which you can submit email for delivery. My home server is my mail server, but you could use your ISP’s smtp server. Another problem is that msmtp just fails if it can’t connect to the mail server – it’s up to you to handle that and do something else with that precious message you can’t mail just now! My sample scripts do not deal with that situation.

This very simple script I call sendmail, and it will need customising for your setup…

#!/bin/sh## set these for your setup...MailServer=mail.serverDomain=MachineNameFrom="me\@my.domain.com"exec msmtp --host=$MailServer --domain=$Domain --from=$From $*

This is my simple script to emulate the simple parts of the send functionality in the BSD mail utility. It has many shortcomings, but it has served me well…

#!/bin/sh## A sort of shell replacment for the send functionality of# the standard "mail" utility.## mail [-s subject] recipient(s)#u=`id -un`d="my.email.domain"r=""s=""v=""while [ $# -gt 0 ]; do p="$1" shift case "$p" in -s) s="$1" if [ $# -gt 0 ]; then shift ; fi ;; -v) v="-d" ;; -*) echo 1>&2 "Option \"$p\" not recognised." exit 1 ;; *) r="$r , $p" ;; esacdoneif [ "$r" = "" ]; then echo 1>&2 "No Recipients." exit 1fiif [ "$s" = "" ]; then printf "Subject: " read sfis="Subject: $s \n"(echo -e "From: ${u}@$d \nTo: $r \n$s \n\n" cat ) | sendmail -t $v

So if you have a job to run on the platform, then this will email the output to you…

my_job | mail -s “my_job output” [email protected]

msmtp can be loaded from the package systems of most distributions, but I have had occasion to cross-compile the package for installing on a system without package management. I had only limited libraries on my cross compile system, and found that after downloading and extracting the sourcecode from sourceforge, I had to cross compile without some of the advanced features. I used

./configure –build=arm –disable-ssl –disable-gsasl –disable-nls

before doing the make to build the binaries. The resultant binary just submitted plain text email, but that was ok for my use – YMMV. The resultant binary, suitably stripped,is pretty lean.

I recently revisited using msmtp to pre-test a change to my ISP’s new smtp server, before committing the change to my mailserver’s sendmail setup. It can also be useful for testing security settings etc on mail submissions systems.

Jim

READ MORE HERE

Handy Terminal Keyboard Shortcuts

Posted by Derek@TheDailyLinux in Tips and Tricks » Add Comment » 

Put these into your “Terminal Guru” belt and be more productive!

Cursor Movement Control
Ctrl-a: Move cursor to the start of a line
Ctrl-e: Move cursor to the end of a line
Ctrl-Left/Right: Navigate word by word (may not work in all terminals)

Modify Text
Ctrl-w: Delete the whole word to the left of the cursor
Ctrl-k: Erase to end of line
Ctrl-u: Erase to beginning of line

Scrolling/Buffer Control
Shift-PageUp/PageDown: Scroll through current buffer
Ctrl-s: Pause terminal output (program will keep running)
Ctrl-q: Release terminal output (after being paused)
Ctrl-l: Clears the screen. Use this instead of the clear command.

History
Ctrl-r: Search the history (enter to run the command once found)

Bonus Tip: Use ‘!!’ command to run last command and ‘!com’ to run the last command starting with ‘com’.

Process Control
Ctrl-d: Exit
Ctrl-c: Kill the current process
Ctrl-z: Put the current process in the background (fg will restore it)

Are there any keyboard shortcuts that you can’t live without? Tell us about them in the comments below.

READ MORE HERE