PLUMI

Installation

This is a guide for installing Plumi 4.x on a Debian or Ubuntu Linux server. You can either use the nginx web server that's automatically built and configured by the Plumi buildout, or you could manually configure an Apache2 web server instead.

Plumi is untested on other operating systems for production. You can install the development buildout in MacOS X as well. More information about installing Plone on different kinds of servers can be found here, which should aid you in installing the Plumi buildout on other operating systems. Please contribute your installation notes on these or different OSs via the Plumi email list and/or by editing this manual chapter here on FLOSSmanuals.

Prior Knowledge

This guide requires that you already have basic knowledge of Linux systems administration. Basic knowledge of Python, Plone and Zope is desirable but not required for installation.

Requirements

Debian or Ubuntu Linux server. It's highly recommended to have at least 2GBs of RAM and two or more processor cores. Also check out the Plone system requirements
Shell access (ftp is not enough!)
Root access for production deployments, also necessary for the initial installation of the dependencies in development builds.

Plumi 4.x is built on Plone 4.x which requires Python 2.6. It will be installed as part of the installation detailed in this chapter along with a series of tools like gcc, make, subversion and more.
Two hostnames required for production deployments. You should configure those to point to your server's ip address.

One for the Plumi site itself (e.g. www.example.org)
A second one for the transcode daemon and the video server (e.g. videos.example.org)

NOTE: If you server is not going to be accessed from the Internet, you still need to set up the hostnames within your private network (eg: internal DNS server, /etc/hosts files on all workstations, etc). If you want to run a development version to test plumi, you do not need to set up these domains. Plumi development setup will work out of the box on localhost.

Setting up dependencies

You can install Python and other required tools by running these commands, either as root or by using sudo (e.g. sudo apt-get install ...).

gcc/g++ dev tools and yasm

apt-get install build-essential yasm

subversion

apt-get install build-essential yasm

python2.6 and header files

apt-get install python2.6-dev

libjpeg and zlib header files

apt-get install libjpeg62-dev zlib1g-dev

groff-base

apt-get install groff-base

???

sudo apt-get install libssl-dev python-profiler
sudo apt-get install libreadline5-dev libxml2-dev

Installing via Buildout

Here you will find information on how to install Plumi from buildout, which is the recommended process. Buildout puts together Zope, Plone and the third-party products required by Plumi like ffmpeg, nginx, varnish, haproxy and more. For more info on buildout check out the Plone.org buildout tutorial.

Before installing Plumi, you need to answer the following questions:

Where are you going to put your Plumi files on your web server?

We will be using /opt/plumi for this guide but you could use /home/user or something similar.
NOTE: Do not install the files into /var/www or any other publicly accessible space on your server.

Which version of Plumi do you want to get?

The trunk version of Plumi contains the latest development code and may therefore become unstable. If you are contributing to the development of Plumi by coding or testing then this is the version you should install.
Tagged versions of Plumi are releases that have been packaged and may include Beta Releases, Release Candidates or full production versions.

Do I want to create a development site or a production site?

If you wish to run a production site you can use the fully-fledged production build with caching, multiple load balanced zope instances, etc.
If you simply want to evaluate or develop Plumi, then a much more lightweight buildout is provided instead in the development buildout.

Which ports do you want Plumi to use?

By default, Plumi uses the following ports

8080 to run the Zope development instance
8100 to run ZEO (the object database server)
8888 for the Transcode Daemon
8890 for Supervisor that manages all the different services
8081-8088 & 8091-8098 for the Zope instances (only for production buildouts)
8090 for the load balancer (only for production buildouts)
8021 for the Zope ftp server (only for production buildouts)
8099 for the Varnish cache server (only for production buildouts)
21 for the Plone FTP proxy (only for production buildouts)
80 for the nginx web server (only for production buildouts)

You can see and change all the ports used by editing the site.cfg file. If you have other systems using these ports, change them prior to running buildout (see the buildout configuration section for details).

Under which system user accounts should the different Plumi services run?

This is only relevant for production builds. If you're running a development build you can start all processes as your own system user. This is possible for production builds as well, apart from the web and ftp servers that normally listen on privileged ports (80 and 21 respectively). However for public websites it is hilghly recommended that different services should run under different system users for improved security.

The users need to exist in your system. If not make sure you create them (e.g. sudo adduser zope). To change the user mappings edit the respective options in site.cfg. By default, the production buildout will start the following services as the respective users below:

www-user = www-data
ftp-user = daemon
transcode-user = daemon
zeo-user = zope
zope-user = zope
balancer-user = proxy
cache-user = proxy

How many load balanced Zope instances do you need?

This is only relevant for production builds. Depending on your site's traffic you may choose to enable up to 8 load balanced Zope instances. 2 instances should be enough for sites without significant traffic, 4 for medium traffic and 8 for high traffic websites. The more instances, the more concurrent users can be served by your site. Beware that more active instances require more RAM but can also utilize more cpu cores. If RAM is not an issue you should have as many Zope instances as the available cpu cores in your server.

By default only 2 instances are enabled. To enable more edit the active-instances option and comment in instances 3-8.

Buildout Installation Process

This process assumes that you are creating a Plumi site called "plumi" at www.example.com with the video .

Download Plumi from SVN

Plumi is available from the Plone SVN server. To download the latest development version of Plumi, run the following command:

svn co https://svn.plone.org/svn/collective/plumi.app/trunk/ target_folder_name

To download the latest Tagged version of Plumi, run the following command:

svn co https://svn.plone.org/svn/collective/plumi.app/tags/Tag_Name  target_folder_name

NOTE: target_folder_name is where you want the code to go (eg: /opt/plumi/4.0-final or /home/zope/plumi) and Tag_Name is tagged version you want to get (eg: 4.0, etc)

Download Plumi as a Packaged Release

Plumi can be downloaded as a fully packaged release in an archive using the tar or zip format. Only tagged versions are available this way, not the current trunk. To download a packaged release, visit the Plone product page here:

http://plone.org/products/plumi/releases/

Find the release you want, copy the download link and follow the instructions below e.g.:

mkdir /opt/plumi
wget Download_Link
tar -xzvf Zip_File
cd Created_Directory

NOTE: Download_Link is the link to the package file from Plone.org / PyPI e.g. Zip_File is the name of the archive file (eg: plumi.app-4.0.zip) and Created_Directory is the name of the directory created after the archive is extracted (eg: plumi4.1beta).

Creating a virtualenv (optional)

If you are running other Python applications on your server or development workstation, it may be worth while to create a virtual Python environment for your Plumi build. This can prevent conflicts between Python packages installed by your operating system and those installed by the Python setuptools or the pip installer. You will need to have the virtualenv package installed first - in Debian and Ubuntu systems you can apt-get install python-virtualenv, then:

$ virtualenv -p /usr/bin/python2.6 --no-site-packages target_folder_name

$./target_folder_name/bin/activeate

This will create and activate a new virtualenv named target_folder_name which will not share any Python packages with the rest of your system.

Set Site Configuration Information

If you're running a development build this step is optional. If you skip it you will be accessing your plumi site at http://localhost:8080/plumi while transcodedaemon will be listening at http://localhost:8888.

For production builds you should configure the specific information about this Plumi installation in the site.cfg, including:

Change www-server-name and www-videoserver-name to the public hostnames of your website and your transcode daemon/videoserver
Check the ports and change them if the defaults are already in use
Set the effective users the respective processes as described above (NOTE: These user accounts should already be set up - the Plumi installation script will not create them)
Set the default Zope admin user & password if you want them to be different (recommended)
Modify the secret key for secure transcode requests

Devel buildout : Run the buildout then setup & configure Plumi

Prepare to run the buildout (bootstrap is a Python script that creates a number of directories and scripts based on the buildout configuration file, and downloads the latest version of the zc.buildout egg).
```
$ python2.6 bootstrap.py -c devel.cfg
```
Run the buildout. This will take some time depending on your server's configuration & load. Sit down, grab a coffee and wait till it finishes!
```
$ ./bin/buildout -c devel.cfg
```
Supervisor will be managing the ZEO server and the transcode daemon. Start supervisor like this

  ./bin/supervisord

./bin/supervisorctl status

./bin/instance-debug fg

Production buildout : Run the buildout then setup & configure Plumi

Prepare to run the buildout (bootstrap is a Python script that creates a number of directories and scripts based on the buildout configuration file, and downloads the latest version of the zc.buildout egg).
```
$ python2.6 bootstrap.py -c production.cfg
```
Run the buildout. This could take quite a long time, depending on your server's configuration & load. Sit down, grab a coffee and wait till it finishes!
```
$ ./bin/buildout -c production.cfg
```
Supervisor will be managing the ZEO, Zope instances, load balancer, cache server, the ftp proxy and the transcode daemon. Start supervisor like this
```
sudo ./bin/supervisord
```
You can check that all the services have started via :
```
sudo ./bin/supervisorctl status
```

Production buildout sets up and configures an nginx web server based on your site.cfg setup. However, if your server already has Apache set up on it (eg: for hosting other, non-Plumi sites), you can stop nginx and use configure apache to use with plumi. Here is the apache configuration for demo.plumi.org to help get you started:

<VirtualHost *:80>
        ServerName  demo.plumi.org
        ServerAdmin contact@engagemedia.org

        <Directory />
                Options FollowSymLinks
                AllowOverride None
                Order allow,deny
                allow from all
                deny from 60.224.93.59,203.184.26.179
        </Directory>

        ErrorLog /var/log/apache2/plumidemo/plumidemo-error.log
        # Possible values include: debug, info, notice, warn, error, crit,
        # alert, emerg.
        LogLevel warn

        CustomLog /var/log/apache2/plumidemo/plumidemo-access.log combined
        ServerSignature On
        RewriteEngine On

 # Normalize URLs by removing trailing /'s
 RewriteRule ^/(.*)/$ http://127.0.0.1:8381/VirtualHostBase/http/%{SERVER_NAME}:80/demo/VirtualHostRoot/$1 [L,P]
 # Pass all other urls straight through
 RewriteRule ^/(.*)$  http://127.0.0.1:8381/VirtualHostBase/http/%{SERVER_NAME}:80/demo/VirtualHostRoot/$1 [L,P]

  <Location />
   Order Allow,Deny
   Deny from none
   Allow from all
  </Location>

</VirtualHost>
<VirtualHost *:80>

    ServerName videos.demo.plumi.org

    RewriteEngine On
    RewriteLog /var/log/apache2/demo.plumi.org/rewrite_videos.demo.plumi.org.log
    RewriteLogLevel 0

    CustomLog /var/log/apache2/demo.plumi.org/access_videos.demo.plumi.org.log common
    ErrorLog /var/log/apache2/demo.plumi.org/error_videos.demo.plumi.org.log

    <Proxy http://127.0.0.1:8338>
      Allow from all
    </Proxy>

    #Alias /transcoded /var/log/apache2/demo-41.plumi.org/transcoded
    Alias /transcoded /home/plone/demo-41.plumi.org/transcoded
    <Directory /home/plone/demo-41.plumi.org/transcoded>
       Order allow,deny
       Allow from all
    </Directory>

    RewriteRule ^/transcoded - [L]
    RewriteRule ^(.*)$ - [E=BACKEND_LOCATION:127.0.0.1]
    RewriteRule ^(.*)$ - [E=BACKEND_PORT:8338]
    RewriteRule ^(.*)$ - [E=HOST:videos.demo.plumi.org]
    RewriteRule ^(.*)$ - [E=PORT:80]
    RewriteRule ^(.*)$ - [E=PROTO:http]
    RewriteRule  ^/(.*)/$ http://%{ENV:BACKEND_LOCATION}:%{ENV:BACKEND_PORT}/$1 [L,P]
    RewriteRule  ^/(.*)$ http://%{ENV:BACKEND_LOCATION}:%{ENV:BACKEND_PORT}/$1 [L,P]

</VirtualHost>

Once the instance is running, go to the Zope Management Interface (ZMI).
http://www.example.org:8080/manage

If you've used a non-standard port for Zope, replace the 8080 in the URL above with the port you have specified when configuring the buildout.

Log in to the ZMI using the admin login details you specified in site.cfg (eg: username: admin, password: admin). If you want to change the admin user's ZMI password once logged in, navigate to the acl_users folder in the left-hand sidebar.

Click on users.

Click password.

Enter password and confirm by re-entering, then click Update Password.
Add a Plone site or instance in the ZMI.

Click “Root Folder” in the left-hand sidebar. Choose Plone site from the Select type to add... drop-down menu and click Add.

Enter the ID, Name and Description, e.g. ID "plumi" and Name "Plumi Site". Click Add Plone Site.
Visit your new plone site by entering this URL: http://www.example.org:8080/plumi

If you've used a non-standard port for Zope, replace the 8080 in the URL above with the port you have specified when configuring the buildout.

Click site setup in the top right corner.
Follow the prompt to change your mail settings.
Change/add your mail settings and click Save.
Click Add/Remove Products from Site Setup sidebar.
Tick the tick box next to Plumi.
Click Install.
Wait a few minutes for your Plumi site to install.
Click Add/Remove Products from Site Setup sidebar again.
Tick the tick box next to Plumi skin.
When it is done, the page will reload and you should see a freshly installed Plumi site complete with the Plumi skin/theme.
Add a new user to your Plumi site for testing (the admin user may not have the correct folders created for them, as this user is created before the Plumi site is generated).

Click log out.

Register a new user by clicking register.

Log in with new user account details by clicking log in.
Try publishing a video in your new Plumi site (logged in as this new testing user).
Check the video after it has uploaded to see if it has been created and transcoded correctly.

Configuring Plumi to start automatically

To ensure that Plumi will start automatically whenever your server is turned on, add the following lines to your /etc/rc.local file:

./bin/instance start
./bin/squid-instance
su zope -c './bin/transcodedaemon start'
./bin/apachectl start

NOTE: Replace the "zope" in "su zope -c" with the name of the user you installed Plumi under (the "effective-user" from site.cfg)

Troubleshooting

If you are having trouble installing Plumi please join one of our email lists or IRC channels listed here or contact the Plumi maintainers here.

Running Multiple Plumi Buildouts

You may encounter difficulties if you are already running a Plumi site from another buildout on this webserver (note that you would usually add another Plone site within the one buildout rather than run multiple sites from different buildouts). If you must run multiple Plumi installations on the same server, you will need to set up Apache & Squid yourself & not use the default configurations included in the buildout process. You will also need to change the default ports in the installation's site.cfg file. If you are running more than one transcode daemon as well, you will need to change the name of the Transcode Daemon process ID file.

Development Map

The development map chapter describes the anatomy of the Plumi buildout, and explains the products and packages that are installed.