zircote.com

development (in)action

Access Control by IP Address Ranges

Simple example of controlling access to resources based on origin IP address, this example validates origin is a private IP range.

I have also added the supporting unit test, note that I am reflecting the method inorder to set it to public visibility to allow testing of it. It should also be note that in order to generate the tests you must change the protected to public so that PHPUnit will recognize the annotated assertions and generate the desired tests.

Zend_Acl Storage With Redis

Recently I have been playing around with Rediska and Zend_Acl looking at different ways to store the lists and manipulate them so I put together a few of my scribbles to share.

Requirements:

This basic demonstrate support for multiple containers, utilizes a singleton for creation of each container. The creation of nonexistent acl containers within redis is delegated to the Rediska_Key::getOrSetValue method for initialization of the redis key being requested. This method uses the Acls name (‘BugsService’) through ReflectionClass to locate the proper creation controls.

SomeAclService::getInstance($aclName) returns the RediskaAcl container with name: $aclName.

RediskaAcl extends Zend_Acl, providing methods: getAcl() returns the object, creating the object and saving to the redis namespace if it does not exist setAcl(), allows a manual save of the current Acl in the event it is modified. setRedis() allows for run time Rediska instance configuration.

Model_BugsServiceAcl provides the mechanisms by which first time creation of the Acl is realized, this may be a Mapper to the database models that would generate the Acl and return the RediskaAcl object that is then saved to the redis namespace.

RedisAclTest demonstrates the use and verification of proper functionality of the Acl roles and resources.

Finally a short example of runtime usage

Testing Web-Service Clients With Zend Framework

It seems the majority of my time these days is spent writing clients to consume all of the fantastic web-services that people and companies create for us, we are the fortunate beneficiaries of these peoples efforts. Developing clients to consume web-services has at times the additional sticky points that other efforts development may not, there may be a  case were we can not afford to make actual real-live calls to an endpoint (no test sandbox, monetary charges or we simply do not possess actual access to the endpoint.) In these case and most especially while testing we do not want to make ‘real-live’ calls to the endpoint. It is not nor should it be the purpose of our tests to test the endpoints performance. A functional test of an endpoint is unrelated to the client libraries we have developed and wish to test and as such should never be dependent on the endpoint for testing its performance characteristics.

I hate reinventing the wheel for common problems that many very smart people have already solved and solved well; which is why most of my personal development is based on a framework, specifically Zend Framework.  The Zend Framework has great support for testing in a vast majority of its components and for the purpose of this article we will be utilizing the Zend_Http_Client for the examples.

The strategy for testing web-services is to simply mock the Zend_Http_Client_Adapter_* the request employs; each test will decide in the setup of the test what status code will be sent, what headers and what the body content will contain. The approach I take with this is creating static files within a fixtures directory ( tests/fixtures/http/someEndpoint.200.xml),  making an effort to identify each documents purpose and how it should be used in the test. I may include in these files the any number of unwanted errors, malformed documents entities in an attempt to expose weaknesses in my handling, (this is especially handy when bugs are discovered that are undocumented in the provider API or unforeseen in development):

Examples of the http response status and content-type should reside in strings similar to the following (note additional headers may be appended following this forming the expected document preamble:

The test belows demonstrates the key points of setting up the test; the setup method provisions the class being tested which in this case extends the Zend_Http_Client. This provides a method setAdapter; with this method we are able to provide the Zend_Http_Client_Adapter_Test for use in our testing. We will test one method, accountStatus and are only demonstrating one test case testApiSuccess within this test case we read in the fixtures file contents and append them to the responseHeader we declared in the #setUp method; this string is passed to the call:$this->Api->getAdapter()->setResponse($fixture); and becomes the desired result of our web-service call. This process of mock returns affords us the ability to generate infinite result sets across a varying range of expected or anticipated failure scenarios.

Adding DocBlox to Your Continuous Integration

Recently inspired by Sebastian Bergmanns, jenkins-php.org I have begun making a push to put all of my projects both personal and professional under continuous integration management; however one project I work with was so large it was impossible to complete the build. This lead me to discover DocBlox: ”a Documentation Generation Application (DGA) for use with PHP applications.” It was built with this problem in mind and is gaining rapid adoption throughout the PHP community.

Integrating DocBlox in your Jenkins instance is simple and straight forward implementing the changes are as follows:

Discover and install DocBlox.

*note DocBlox requires libxslt support within PHP as well as http://www.graphviz.org/.

Convert your build.xml to run docblox.

 

Capturing Ec2 Instance Meta-data

A simple class to grab the meta data of an EC2 instance for shell scripting as well as use in other actions on the server. I utilize it as part of the cluster member adding and deleting when autoscaling instances.

Starting a Zend Framework Project [2/2]

Previously I discussed the creation of a ZF project skeleton; having established the test suite, build file, phpdoc and phpcs processing, we can now begin building the database models. I will also expand on relationships within the models and will cover the implementation of model relationships using Zend_Db_Table_Relationship.

The Database

For the purpose of this entry, I will utilize the database model described in the Zend Framework documentation; this database contains four tables:

  • accounts
  • products
  • bugs
  • bugs_products

To create the database follow, the commands outlined below. Be sure to ‘cat’ the contents of the following gist into db/development.sql:

Now that your database is created we may generate the table models.

Table Models

To generate table models, we will employ the zf cli providers; it should be noted that while it is possible to create them by hand this does not maintain the manifest. .zfproject.xml . The commands to create the models are as follows:

It should be noted that additional parameters are available. The list of parameters are as follows:

  • name: the models name, this will be in addition to the namespace portion; ie App_Model_DbTable_
  • actual-table-name: the table name within the database
  • module: the module the model will reside, in this case default in which case we submit nothing
  • force-overwrite: whether to rewrite the model

It should be noted that commands in the ‘zf’ project provider will also accept parameters in the following manner -f force-rewrite -m module -n name -a actual-table-name

Table Relationships

The Zend_Db package also provides the tools to create and maintain relations between models enabling a series of methods that allow for child and parent queries from result row sets. As the db models are extended from Zend_Db_Table there are two properties of interest, Zend_Db_Table::dependentTables and Zend_Db_Table::referenceMap.

We will begin with the ZendDb_Model_DbTable_Accounts and declare our dependant table, ZendDb_Model_DbTable_Bugs as an array member of the ZendDb_Model_DbTable_Accounts::referenceMap property. Following the ZendDb_Model_DbTable_Accounts model, we will next detail the ZendDb_Model_DbTable_Bugs model and associated references, and dependencies. Within the bugs table three columns are dependencies of accounts table, reported_by, assigned_to, and verified_by each of which references accounts. This is represented in the model ZendDb_Model_DbTable_Bugs::referenceMap as follows:

For the cascading actions for Zend_Db_Table::_referenceMap there are three class constants provided for declaration:

  • Zend_Db_Table::CASCADE
  • Zend_Db_Table::RESTRICT
  • Zend_Db_Table::SET_NULL

Additional details maybe found at:

Zend_Db_Table_Definition

As an alternative to defining relationships within each model there is another option, Zend_Db_Table_Definition for this projects database definitions. I would prefer to maintain the definitions within the application.ini and utilize the resource autoloader to place it within a Zend_Registry container (note: it is possible to utilize a separate .ini file for the database definition.) It will also be necessary to change the models to extend from the Zend_Db_Table instead of Zend_Db_Table_Abstract , in order to gain the constructor functionality required to define the definition and key specific to each model. Within the models we may now define our database definition as well as the definition key in the constructor. To begin the changes required to enable the table definitions, we first must create the resource entries within the application.ini. These definitions are laid out in the .ini in the same manner as an array. Note the first keyname (‘resources’) is required so the bootstraper will seek to use it to find the bootstrap method within the Bootstrap class.

Next we create the resource bootrap method within the Bootstrap.php. The goal of this is to load and instantiate the Zend_Db_Table_Definition object and store it within a Zend_Registry container for use by our models.
Next, we must modify our models to reflect the changes, namely the constructor. Within these models constructors we pass the definition key that corresponds to the class in question as the first parameter; with the second parameter, we will pass the definition object which we will acquire from the Zend_Registry container created within the bootstrap. We must do this for all database models.
Having updated the models, application.ini and Bootstrap.php examples of their magic methods, as well as explicit calls, may be found in the Zend Framework reference guide; I have also provided the following snips to demonstrate explicit calls:

Starting a Zend Framework Project [1/2]

Recently I have been asked to introduce and assist with starting Zend Framework projects; over time I have learned through trial, error and banging my head against a wall a few tricks, shortcuts and nuances. In this series I will take the life cycle of a Zend Framework project from beginning to never-end and cover each portion as completely as I can and provide a place for discussion to improve my process or detail it further. It should be clear the intent of this article is not to discuss development methodology or any process outside of the scope of using Zend Framework itself.

A project is born.

Beginning with a name for the project established (for the purpose of this write-up I have decided on ZendDB) we should begin by creating the project reference within Eclipse, Zend Studio or your preferred editors method. Before we begin any code creation or development we need to insure we have the required tools on our system to begin.

Following this we must establish where we will edit, build and work on the project and initialize the project contents using the zf tool, create additional directories for build and validation.
The build related files and directories tie all components into a collective suite of tools, utilizing ant or phing we will be able to execute all reports as one single command. This practice also lays the foundation for continuous integration, which this project will evolve into as time goes on. To enable each of these tools we will use we must create their associated configurations, begin by editing build.phpcs.xml
I have intentionally kept this example simple and to the point, I strongly recommend you visit the PHP_CodeSniffer project for more details on its capabilities. Once complete the PHPUnit configuration is setup with the various types of reports it should generate, the test suites it will cover.
For more details on PHPUnits report capabilities and xml configuration options visit the PHPUnit Documentation. Now this is established we may bring it all together with the ant build.xml file. This file will allow us to create/remove required build artifacts, report directories and execute phpunit, phpcs and phpdoc all in a single stack.
For a more detailed overview of an ant build file please see Apache Ant; it should also be noted that Phing is a php alternative to ant and should be interoperable with ant. To utilize the build script you need only execute ant within the top-level directory of the project, the example file provided will create phpdocs, code sniff the project and execute all unit tests. Reports will be stored within build/* for review.

In the next edition to this I will cover database configuration and model creation.

Cleaning Up Your Product

I recently made the embarrassing mistake of creating a push request to a well respected PHP author’s project and to my horror I saw he had to ‘detabify’ my push. I had recently reset my Zend Studio workspace and neglected to reconfigure the default format standard from PHP => Zend.

Once I sorted out my studio client, I began to look into ways to test this all before it’s even committed, to insure I have not overlooked a detail. To help with syntax, formatting and style, I have employed PHP_CodeSniffer, a PEAR package. Its use is simple and straightforward with informational documentation; Rulesets or Sniffs are varying and customizable in the form of a ruleset xml file. Available Sniff Collections are grouped as follows:

Each brings its own standards ruleset for customization. I have chosen to implement the Zend Sniff in  the form of a file named phpcs.xml, I exclude certain file types .js, .css all test files, data files and, for good measure, .svn directories.

I place this file in a directory inside /tests/build/ where you place your file strictly depends ons on your personal preferences. To install PHP_CodeSniffer execute the following commands:

There are several choices of reports, for the purpose of command line reporting I choose one or all:

  • source [provides a description of what rules the violations are related]
  • full [detail of all violations in the code base, line of violation and rule
  • summary [ a high level assessment of all rule violations]

Examples of the report type’s outlines and the associated commands to produce each with the afore mentioned rules file.

[gist id=”965105”/]

While it’s possible, the report will be sizable depending on the projects size; however this is an opportune time to begin cleaning up your existing code base; but more importantly, it gives you the tools required to insure your code is clean, consistant and within community expectations.

If you, your project or employer have your own specification of code standards, I would encourage you to create your own rule set in combinations. Should the existing sniffs not be sufficient, it’s advised to create your own sniffs to accommodate your needs.

The PHP_CodeSniffer project provides an annotated xml ruleset file to serve as a guideline and tutor; as well as the documentation associated with its use.

Skulk PEAR Release 0.1.2

I am happy to announce the release of Skulk 0.1.2 for use with Prowl. This release introduces a command line zf provider for your messaging pleasure. More on its use and source code may be found at github@Skulk.

For PEAR installation and channel information, please see: http://pear.zircote.com/ for more details.

Setting up and using the Provider

Create the user config file for Zend_Tool

zf create config 
It will be located at ~/.zf.ini and can be opened in your preferred editor. Insure the Skulk library directory is in the include path, then execute the following:
$ zf enable config.provider Skulk_Tool_ProwlProvider Provider/Manifest 'Skulk_Tool_ProwlProvider' was enabled for usage with Zend Tool. 

You are ready to set up the configuration and send a message.

$ zf ? prowl Zend Framework Command Line Console Tool v1.11.5 Actions supported by provider "Prowl" Prowl zf addkey prowl keyname apikey zf delkey prowl keyname zf send-message prowl message priority[=normal] description url 

Add your API key(s) with the following command:

zf addkey prowl   

Now send a message:

$ zf send-message prowl 'test message' 988 messages left until May 8, 2011 5:58:48 PM

 

Writing a Custom Provider

I decided I wanted to have a CLI interface to my Skulk library and provide a method to send messages from scripts without requiring any additional coding overhead. To do this I employed the Zend_Tool CLI [zf] and began by deciding on the functionality it would possess. After some thought, I settled ultimately that sending messages would be sufficient for the time being; however, it would require some supporting methods for api key management, naming, adding and deleting.

class Skulk_Tool_ProwlProvider extends Zend_Tool_Project_Provider_Abstract
{
}

I started the process by creating the provider class Skulk_Tool_ProwlProvider; the “Provider” portion of the name is important as the IncludePathLoader will not otherwise recognize the class as such and extending the Zend_Tool_Project_Provider_Abstract.

require_once ('Zend/Tool/Project/Provider/Abstract.php');
class Skulk_Tool_ProwlProvider extends Zend_Tool_Project_Provider_Abstract{
    public function addkey($keyname, $apikey){
    }
    public function delkey($keyname){
    }
    public function sendMessage($message, $priority = 'normal', $description = null, $url = null){
    }
}

Having established the class we may now move on to the public methods. We have established our required methods as follows:

  • Adding an api key
  • Removing an apikey
  • Sending a message
require_once ('Zend/Tool/Project/Provider/Abstract.php');

class Skulk_Tool_ProwlProvider extends Zend_Tool_Project_Provider_Abstract{

    public function addkey($keyname, $apikey){
        if(!$this->_registry->getConfig()->skulk){
            $skulk = array(
                'defaults' => array( 'priority' => 'normal'),
                'keys' => array($keyname => $apikey)
            );
            $this->_registry->getConfig ()->skulk = $skulk;
            $this->_registry->getConfig ()->save();
        } else {
            if($this->_registry->getConfig()->skulk->keys){
                $this->_registry->getConfig()->skulk->keys->$keyname = $apikey;
            } else {
                $this->_registry->getConfig()->skulk->keys = array();
                $this->_registry->getConfig()->skulk->keys->$keyname = $apikey;
            }
            $this->_registry->getConfig ()->save();
        }
    }
}

To add the API key we will need to be able to write to the configuration file in the user’s home directory or where otherwise specified at runtime. This will require interaction with the internal registry property; testing if the relevant properties exist in the configuration, modify it in the event it does or initialize it if not. We perform the various configuration operations through the _registry property as demonstrated in the example below adding the required options followed by the save method to write the configuration file.

$this->_registry->getResponse()
    ->appendContent('api key saved!', array('color' => 'green'));

We also may want to provide feedback to the user regarding the status of the operation. This also is provided through the access of the _registry property with the response object, which may be formatted for color, tab depth, centering and blocking. For the purpose of this demonstration, we will use the color decorator to make the output green, leaving our example well established and ready for the next steps.

    public function addkey($keyname, $apikey){
        $skulk = $this->_registry->getConfig()->skulk;
        if(!$skulk){
            $this->_registry->getResponse()
                ->appendContent('initializing default config for Skulk...', array('color' => 'cyan'))
                ->appendContent('default priority [normal]...', array('color' => 'cyan'));
            $skulk = array(
                'defaults' => array( 'priority' => 'normal'),
                'keys' => array($keyname => $apikey)
            );
            $this->_registry->getConfig ()->skulk = $skulk;
            $this->_registry->getConfig ()->save();
        } else {
            if($this->_registry->getConfig()->skulk->keys){
                $this->_registry->getConfig()->skulk->keys->$keyname = $apikey;
            } else {
                $this->_registry->getConfig()->skulk->keys = array();
                $this->_registry->getConfig()->skulk->keys->$keyname = $apikey;
            }
            $this->_registry->getConfig ()->save();
        }
        $this->_registry->getResponse()
            ->appendContent('api key saved!', array('color' => 'green'));
    }

Now to add the delkey method using the same tools as before, with the addition of an unset on the existing property followed by the save method, as in the addkey logic writing the file with the updated values.

    public function delkey($keyname){
        $config = $this->_registry->getConfig();
        if($config->skulk->keys->$keyname){
            unset($config->skulk->keys->$keyname);
            $this->_registry->getConfig ()->save();
            $this->_registry->getResponse()
                ->appendContent($keyname . ' apikey removed...', array('color' => 'red'));
        } else {
            $this->_registry->getResponse()
                ->appendContent($keyname . ' apikey does not exist...', array('color' => 'red'));
        }
    }

Finally this brings us to the sendMessage call, which will require ancillary methods to create the outgoing message container, the API object that sends the message as well as a priorities method to convert users input into a value expected by the API actor. The details of which are less about Providers and more about the code being employed, leaving us with a final product ready to send messages to an iOS device near you.

require_once ('Zend/Tool/Project/Provider/Abstract.php');

/**
 * @author zircote
 *
 * zf enable config.provider Skulk_Tool_ProwlProvider
 * zf create config
 * zf ? prowl
 */
class Skulk_Tool_ProwlProvider extends Zend_Tool_Project_Provider_Abstract
    implements Zend_Tool_Framework_Provider_Pretendable{
    /**
     *
     * @var Skulk_Client_Message
     */
    protected $message;
    /**
     *
     * @var Skulk_Client
     */
    protected $api;
    /**
     *
     * @example zf addkey prowl iPhone sdf234g9i24t09j23r...
     * @param string $keyname
     * @param string $apikey
     */
    public function addkey($keyname, $apikey){
        $skulk = $this->_registry->getConfig()->skulk;
        if(!$skulk){
            $this->_registry->getResponse()
                ->appendContent('initializing default config for Skulk...', array('color' => 'cyan'))
                ->appendContent('default priority [normal]...', array('color' => 'cyan'));
            $skulk = array(
                'defaults' => array( 'priority' => 'normal'),
                'keys' => array($keyname => $apikey)
            );
            $this->_registry->getConfig ()->skulk = $skulk;
            $this->_registry->getConfig ()->save();
        } else {
            if($this->_registry->getConfig()->skulk->keys){
                $this->_registry->getConfig()->skulk->keys->$keyname = $apikey;
            } else {
                $this->_registry->getConfig()->skulk->keys = array();
                $this->_registry->getConfig()->skulk->keys->$keyname = $apikey;
            }
            $this->_registry->getConfig ()->save();
        }
        $this->_registry->getResponse()
            ->appendContent('api key saved!', array('color' => 'green'));
    }

    /**
     *
     * zf delkey prowl iPhone
     * @param string $keyname
     */
    public function delkey($keyname){
        $config = $this->_registry->getConfig();
        if($config->skulk->keys->$keyname){
            unset($config->skulk->keys->$keyname);
            $this->_registry->getConfig ()->save();
            $this->_registry->getResponse()
                ->appendContent($keyname . ' apikey removed...', array('color' => 'red'));
        } else {
            $this->_registry->getResponse()
                ->appendContent($keyname . ' apikey does not exist...', array('color' => 'red'));
        }
    }

    /**
     *
     * zf send-message prowl 'test message' normal 'long description here' 'http://zircote.com'
     * @param string $message
     * @param string $priority
     * @param string $url
     * @param string $description
     */
    public function sendMessage($message, $priority = 'normal', $description = null, $url = null){
        $this->_init();
        $this->message->setApikey($this->apikeys->toArray())
            ->setEvent($message)
            ->setPriority($this->getPriority($priority))
            ->setDescription($description ? $description : $message);
        if(null !== $url){
            $this->message->setUrl($url);
        }
        $response = $this->api->add($this->message);
        $result = $response->getResult();
        if(key_exists('success', $result)){
            $this->_registry->getResponse()
                ->appendContent($response->getRemaining() . ' messages left until '
                 . $response->getResetDate(), array('color' => 'cyan'));
        } else {
            $this->_registry->getResponse()
                ->appendContent($result['error']['detail'], array('color' => 'red'));
        }

    }

    /**
     *
     * returns integer value for priorityfrom a simple string
     * @param string $priority
     * @return integer
     */
    protected function getPriority($priority){
        $priorities = array(
            'verylow' => Skulk_Client_Message::PRIORITY_VERYLOW,
            'normal' => Skulk_Client_Message::PRIORITY_NORMAL,
            'moderate' => Skulk_Client_Message::PRIORITY_MODERATE,
            'high' => Skulk_Client_Message::PRIORITY_HIGH,
            'emergency' => Skulk_Client_Message::PRIORITY_EMERGENCY
        );
        return  $priorities[strtolower($priority)];
    }

    protected function _init(){
        $this->apikeys = $this->_registry->getConfig()->skulk->keys;
        require_once 'Skulk/Client.php';
        require_once 'Skulk/Client/Message.php';
        $this->message = new Skulk_Client_Message();
        $this->api = new Skulk_Client;
    }

}

When completed it affords us the functionality list:

$ zf ? prowl
Zend Framework Command Line Console Tool v1.11.5
Actions supported by provider "Prowl"
  Prowl
    zf addkey prowl keyname apikey
    zf delkey prowl keyname
    zf send-message prowl message priority[=normal] description url