Solved thread

This post is marked as solved. If you think the information contained on this thread must be part of the official documentation, please contribute submitting a pull request to its repository.

Phalcon with Docker, boot2docker, Vagrant and Puppet

I'm nearing releasing of my Webird CMS based on a heavily modified Vokuro and I am considering including some new deployment environments with the project to encourage people to try it out. Over the last several months I have considered every new technology to build the stack that I can see myself using for the next several years (or more) and this has included; integration of websockets, a switch to nginx, a CommonJS environment, integration of composer, npm, bower, etc. Now I'm considering upgrading my entire development and deployment stack and there are a lot of new options to explore.

I'll first release the Webird source as on github and then I'll be releasing one or more deployment environments for this. Webird currently requires PHP >=5.5 and Ubuntu 14.04 and the availability of this is rather limited at the moment. I plan on also supporting Fedora 21 when it is released.

So I could really use any suggestions and Phalcon related scripts for Puppet, Vagrant and Docker for a Debian environment. I'm currently thinking that Vagrant with Puppet might be the best way to go but boot2docker seems like it offers something similar. I could see supplying a Vagrant setup for Ubuntu 14.04 and Fedora 21 and also a Docker container environment for those already running one of those two environments.

Also I'll add that a particular person who is very skilled in creating testing systems has expressed in buiding up Webird in this regard. Webird currently creates a very tightly integrated distribution build (that is fairly different than the source build) and I can see great value in using this with Docker for development, testing and distribution.



34.7k

wow

I can't wait for thisor )



44.7k
edited Oct '14

I've made up some documentation. It will have to be good enough for the initial release. Here is the WIP README.md:

Webird full application stack

Webird was created to merge the latest PHP and Node.js innovations into a single application stack.

Phalcon and Composer form the backbone of the PHP side and are used to create a HMVC foundation that offers everything that is expected of a modern PHP server side framework. Webpack assembles the browser resources and solves many problem associated with modern web development by bringing the Node.js popularized CommonJS module system to the browser. Node.js is a requirement for development and is no longer required once a project is built.

Webird is hipster forward facing project that is designed to run on a server OS and browser released in 2014 and later. Currently Ubuntu 14.04 is supported.

Key features of Webird:

  • HTML5 IE11+
  • PHP 5.5+
  • PHP Ratchet websockets that offer read-only access to the PHP session data
  • Google OAuth2 login
  • PHP CLI utilities for many tasks
  • Manage most third party dependencies with Composer, NPM and Bower
  • Single PHP installation script for installing local dependencies or type the commands manually
  • Single command that starts various development processes across PHP and Node.js
  • Webpack (CommonJS) build environment
  • Live reloading (and waiting) CommonJS build environment
  • Mix and match any combination of languages and theming (per file) including; CSS, SCSS, LESS, Stylus, Javascript and Coffeescript
  • Complete integration of gettext .po translation data between the PHP and Webpack (Javascript) environments
  • Create a final PHP and Javascript source protected distribution for deployment to the server
  • AngularJS and Backbone/Marionette Webpack examples

Key components of Webird:

Installation:

Webird uses several Node.js npm packages that must be installed globally with root access. Webird also requires some PHP extensions that are not usually installed.

Install Base Requirements (this will vary across Linux distributions):

Ubuntu 14.04:

# Localization
sudo apt-get install poedit

# Node
sudo apt-get install nodejs nodejs-legacy

# PHP
sudo apt-get install gcc php-pear php5-dev php5-mysql libpcre3-dev libzmq3 libzmq3-dev
sudo pecl install zmq
sudo pecl install libevent

# Install Phalcon PHP extension using one method below:

# Phalcon from source
git clone --depth=1 git://github.com/phalcon/cphalcon.git
cd cphalcon/build
sudo ./install

# Phalcon from Ubuntu PPA
sudo apt-add-repository ppa:phalcon/stable
sudo apt-get update
sudo apt-get install php5-phalcon
Install Requirements (Stage 2):

At this point the initial dependencies have been installed. Now you have two options for installing the remainder of the extensions.

1) To install the next stage of dependencies you may run the following script:

php dev/setup/install.php

2) or instead type the commands manually:

# Change to your local Webird dev directory
cd dev
# Add execute permissions to Webird dev script for less typing
chmod u+x webird.php

# Composer program download
# It is recommended (but not required) that you install composer globally to /usr/local/bin/composer
curl -sS https://getcomposer.org/installer | php
sudo mv composer.phar /usr/local/bin/composer

# Node NPM package global installs
sudo npm install -g coffee
sudo npm install -g bower
sudo npm install -g gulp
sudo npm install -g xgettext-template

# At this point all of the global utilities have been installed and now you will leverage
# the three package managers npm, bower and composer.  Once the initial global setup has been done
# you will be able to setup a new installation by simply running these three commands;

# Note: do NOT run these commands with root access:
composer install
npm install
bower install

Configure Localization tools:

Now all of the Webird hard dependencies have been installed but to be able to edit the localization files you will need to configure the GNU gettext frontend Poedit translation program.

The advantage to using gettext is that it supports a wide variety of languages, multiple plural forms and it can extract the translation strings from many types of source code and templates.

The Phalcon Volt templates will compile to PHP embedded in HTML (phtml) and recent versions of the command line gettext xgettext are able to parse this by default. However, to allow Handlebar templates Poedit will need to be configured to use the Nodejs xgettext-template program that you installed globally via npm. To configure Poedit for Handlebar templates you may view the directions on the xgettext-template page or follow the instructions below:

Poedit Configuration Instructions:

Go to File - Preferences... in Poedit and add a new parser in the Parsers tab:

Poedit parser configuration

  • Language: Handlebars
  • List of extensions...: *.hbs
  • Parser Command: xgettext-template --force-po -o %o %C %K %F
  • An item in keywords list: -k %k
  • An item in input files list: %f
  • Source code charset: --from-code=%c

Usage (Development):

  1. Configure dev.json with local development settings
  2. Create and install nginx configuration: dev/webird.php nginx > nginx_dev
  3. Import database schema located at dev/setup/database.sql
  4. Run server processes: dev/webird.php
  5. Profit

Development files are stored in the dev directory and the system is designed so that you should be able to largely forget about this directory once you get into a work flow. The app folder is designed to contain all custom user code and it should never be written to by any automated script or build environment.

Note: In order to use the Poedit Update feature to extract gettext translation messages you must first build a dist environment. If you fail to first build the dist environment then many strings will show up in the Obsolete Strings tab. If this happens then just hit cancel, but don't worry because regardless the old string translations will be preserved within the .po file. This is currently required since there is no xgettext Volt/Twig parser and so the compiled Volt templates must be accessed for these strings.

Usage (Distribution):

Create dist environment:

  1. Configure ./dist.json with final development settings to override settings from ./dev/setup/dist/config_defaults.json. These two files will be merged to form ./dist/config.json.
  2. Create the dist environment: ./dev/webird.php build

Configure final dist environment:

Warning: At this point it will be assumed that you are inside of the portable dist directory wherever it is now located (or named)

  1. Create and install (location dependent) nginx configuration: ./webird.php nginx > nginx_dist
  2. Import database schema located at ./setup/database.sql
  3. Run server processes: ./webird.php
  4. If something is wrong modify ./config.json and repeated steps 1-3. To make changes more permanent for dist releases you may go back and modify the original dist.json file and then rebuild the dist environment.

The nginx configuration must be rebuilt if the distribution environment directory is moved or renamed. It is recommended to use the webird.php nginx command to rebuild the configuration instead of manually editing the generated nginx configuration. If custom settings are required it is recommended to first modify the ./setup/nginx_template file.

Note: Node.js is no longer a dependency at this point since it is only used to build the browser facing content into static bundles.

Project Structure:

The app directory:

- locale (contains the gettext locale .po files uses by Phalcon and Webpack)
- public (the web server root is pointed here.  The files here should be minimal and call bootstrap files at a lower level)
- theme (theme files to be read as-is and also processed by Webpack)
- phalcon
  - ...
- webpack
  - commons (common code to be run by multiple entry points)
  - entries (specific code entry points)
  - helpers (global helper modules that export a single function)
  - modules (custom commonjs modules)

Compare the app directory to a built dist directory to notice the differences between the code that you will be working with and the final production output. You may also view the build system routine at app/phalcon/modules/cli/tasks/DevTask.php

Note: The dist directory does not contain any Node.js/Webpack related code and everything is in a finalized, optimized and protected form. If Ion Cube has been enabled then the build process will use it to protect the PHP code.

TODO and the WAITING:

At the moment only basic websocket support is supported since Ratchet does not support the WAMP v2 protocol and newer Javascript libraries such as Autobahn|JS are now WAMP v2 only and the older v1 versions don't play nice with the CommonJS module system. Ratchet development stalled out with the WAMP v2 feature, but there is hope since the Thruway team is building upon the Ratchet code base and is hard at work to suport a WAMP v2 protocol. There is much colloborative and blessings between the two projects so this looks positive.

not really interested in all of the front end tools for phalcon.

to answer you original question though, definitely investigate docker(boot2docker is just a windows/macosx vagrant wrapper) instead of puppet/vagrant.

docker and containers seem like they are here to stay and will only gain more momentum.



44.7k

I found a Vagrant and a Puppet Phalcon starter on github.



44.7k

I'm realizing that I'm really not an Ops guy so I'm instead making some pretty easy to understand bash install scripts and functions. I've found that getting started with Puppet is no small matter and that just setting up the Puppet modules is a challenge in itself. So I'm setting up the packages to install modules to install packages to install modules. I'll add that I hope it gets a hell of a lot simpler in the following years.

I'm building my bash install script not to be the most comprehensive possible but instead to be extremely easy to understand so that people can in fact follow them.



6.4k
Accepted
answer

@dschissler

If you are writing bash scripts, you could be writing dockerfiles. If you have done this much work with your project to write all of these custom tools, you might as well look into making it seemless to deploy with docker, which uses bash scripts to deploy instead of puppet configs.

This should get you started: https://crosbymichael.com/dockerfile-best-practices.html

A php-fpm dockerfile would begin with something quite like this:

FROM ubuntu:14.04
# Install PHP5-FPM
RUN \
    apt-get update && \
    apt-get install -y \
        vim \
        curl \
        wget \
        build-essential && \
    apt-get install -y \
        python-software-properties \
        software-properties-common && \
    add-apt-repository -y \
        ppa:ondrej/php5 && \
    apt-get update && \
    apt-get install --force-yes -y \
        php5-cli \
        php5-fpm \
        php5-mysql \
        php5-pgsql \
        php5-sqlite \
        php5-curl \
        php5-gd \
        php5-mcrypt \
        php5-dev

RUN \    
    sed -i "s/;date.timezone =.*/date.timezone = UTC/" /etc/php5/fpm/php.ini && \
    sed -i "s/;date.timezone =.*/date.timezone = UTC/" /etc/php5/cli/php.ini && \
    sed -i -e "s/;daemonize\s*=\s*yes/daemonize = no/g" /etc/php5/fpm/php-fpm.conf && \
    sed -i "s/;cgi.fix_pathinfo=1/cgi.fix_pathinfo=0/" /etc/php5/fpm/php.ini

# Install Phalcon Extension
RUN \
    apt-get install -y \
        libpcre3-dev \
        git \
        gcc \
        make && \
    mkdir       /phalcon && \
    git clone   --depth=1 git://github.com/phalcon/cphalcon.git /phalcon/ && \
    chmod +x    /phalcon/build/install

WORKDIR /phalcon/build
RUN \ 
    ./install && \
    echo "extension=phalcon.so" | cat > /etc/php5/fpm/conf.d/20-phalcon.ini && \
    echo "extension=phalcon.so" | cat > /etc/php5/cli/conf.d/20-phalcon.ini

There are many advantages to using docker and developers find it just as easy to use as ops



44.7k
edited Oct '14

@david-duncan

Here is my ./function.sh file. I'm building just a bit of checking into these functions and setting it up so that I can do a low-tech disable of certain features. I'm working on it ATM.

I do very much appreciate your input. I can see the value in formal provisioners but I'm very unhappy with them so far in their current state. Do you see anything so far that is like "OMG Don't do this with Docker!!!!"?

I might just go to pure Docker after all since I'm running on Ubuntu 14.04 on my Desktop for the next 1.5-2.0 years. Then if need be I can always switch to running my Docker setup inside of a Vagrant VM while I transition. I'm thinking that people who do rock out with Vagrant and Puppet will be able to whip up a Puppet file in very little time - that is if Webird does catch on at all. The real winner are the cutting edge single page application front end tools.

#!/bin/bash

system-package-install()
{
  apt-get install -y gcc git make
  apt-get install -y nginx mariadb-server mariadb-client
  apt-get install -y libpcre3-dev libzmq3 libzmq3-dev php-pear php5-common php5-dev php5-cli php5-fpm php5-mysql
  apt-get install -y nodejs nodejs-legacy
  apt-get install -y gettext poedit potool
}

npm-global-install() {
  npm install -g coffee gulp bower xgettext-template
}

php-confd-ini-path() {
  local sapi=$1
  local ext=$2
  local code=
  if [[ -z "$3" ]];
    then code=20
    else code=$3
  fi

  echo "/etc/php5/$sapi/conf.d/${code}-${ext}.ini"
}

php-settings-update() {
  local setting_name=$1
  local setting_value=$2
  local cli_ini=$(php-confd-ini-path cli $setting_name 0)
  local fpm_ini=$(php-confd-ini-path fpm $setting_name 0)

  echo "${setting_name}=${setting_value}" | tee "$cli_ini" "$fpm_ini"
  return $?
}

php-pecl-install() {
  # Split argument by '-'
  local args=(${1/-/ })
  local name=${args[0]}
  local state=${args[1]}

  # Allow for alpha and beta packages
  local package_name=$name
  [ ! -z "$state" ] && package_name=$package_name-$state

  pecl install $package_name
  if [ $? -ne 0 ]; then
    >&2 echo "Pecl package $package_name install failed."
    return -1
  fi

  local cli_ini=$(php-confd-ini-path cli $name)
  local fpm_ini=$(php-confd-ini-path fpm $name)
  echo "extension=$name.so" | tee "$cli_ini" "$fpm_ini"
  return $?
}

php-phalcon-install() {
  local PWD=`pwd`
  git clone --depth=1 git://github.com/phalcon/cphalcon.git
  [ $? -ne 0 ] && return $?
  cd cphalcon/build
  ./install
  ret=$?
  cd $PWD
  return $ret
}

php-composer-install() {
  # First check if composer is installed
  command -v composer > /dev/null 2>&1
  [ $? -eq 0 ] && return 0

  curl -sS https://getcomposer.org/installer | php
  if [ $? -ne 0 ]; then
    >&2 echo "Composer could not be downloaded"
    return -1
  fi

  mv composer.phar /usr/local/bin/composer
  return $?
}

php-fpm-restart() {
  service php5-fpm restart
}


44.7k

I stayed up all night building a Dockerfile and refactoring scripts to allow them to be used indepedently and with Docker. I had to learn some bash lore to work around the 'sh -c' invocation of the Dockerfile RUN command. I was just about at the end of my rope with Puppet. Docker is just super easy and fast to continually retry the build.

tl;dr I hate Puppet and I love Docker.



44.7k

I've mad a lot of progress with Docker and I have it installing all dependencies and now I'm configuring some things. I need to make it configure for nginx globally and I have the installation of the specific nginx working fine. I'm trying to break up the installation between Docker and shell scripts so that it can be installed either way. I think that there are a lot of advantages to consider Docker as the installation system is being built and I can see how otherwise it would be all over the place with various hacks or massive duplication of the installers.

@schissler how far have you gone with this project I am also very interested in this...

I have been working on a similiar project , same concept as on Vokuro but i made is a full stack CMS... with working FE & BE , I also have docker up to the level were you run everything using Nginx from the docker environment...

:)



44.7k
edited Jan '16

@Ben I haven't been running my project in Docker and I just used it to test the provisioning scripts with a small attempt to getting it near ready. Someone recently forked my project and made some nice improvements and I'll be merging them when I have time. I'm working most of the time in my proprietary code and Webird rests inside of another layer. I am occassionally merging improvements back and forth. If I want to test out a new integration then it helps to work with Webird since it is much simpler and sometimes during actual usage I find that some convention could be improved and it is then ported to Webird.

I would be interested to see how you setup Docker as that is new to me.