Featured Posts

Apple Mail Encryption with GPGMail and OpenPGP I've dabbled with encryption several times over the past few decades, never really getting serious about it.  It started when, in college, I would see that the faculty...

Read more

1Password on Ubuntu 11.10 (Oneiric) First, I apologize for being off-the-grid for so long.  I recently started working for a new company and am deep in the throes of the start-up life.  I started working...

Read more

BuddyPress -- Pages failing to load (404) I was going insane working on a buddypress install for our intranet's tech-blog because every page I attempted to load into Wordpress following the Buddypress install...

Read more

BuddyPress -- Pages failing to load (404) I was going insane working on a buddypress install for our intranet's tech-blog because every page I attempted to load into Wordpress following the Buddypress install...

Read more

Renaming mongodb Columns Today I was putzing around in the geo-spatial collection when I noticed that I had an unhappy over one of the column names within the collection. In the mySQL world,...

Read more

Subscribe

mongodb.findOne() — calling with PHP variables (not literals)

Category : Technical
No Gravatar

So I’ve been doing a lot of work, for work, in MongoDB lately and I’ve learned an awful lot.  Or, depending on your point of view, a lot that’s just awful.

See, there’s not what you could even charitably call a lot of MongoDB documentation to begin with.   If you filter what is available on, oh, say, PHP implementation, well the results just dwindle to something roughly the same size as a tax-collector’s heart.

Here’s the scenario — I’ve been working on adding a mongo abstraction class on top of my base-data abstraction class — whereas said classes are extended by the table-level class instantiation.  This allows me to keep all of my query logic in the middle tier of the class design, generic and administrative functions in the base class, and table-specific stuff in the table class.  So far, so good, right?

Well, I get the mongo constructor running and, like it’s mySQL counterpart, I have an rule in every table constructor that states “if I pass a indexed field and it’s value to the constructor, then instantiate the class pre-populated with that record.”

And that’s where things start to head south.

In my constructor logic, I’m only allowing single-value key->value pairs as constructor parameters with the design intention of getting a record from the db using the pkey of the table/collection.  In other words, you get one column and one column value.  So, if you’re going to instantiate a new user object, you’d probably want to pass-in the primary-key field of a user and that field’s value:

$objUser = new UserProfile(‘email’, ‘mshallop@gmail.com’);   // instantiate a new user object with this email address

Still pretty easy.  I bang out the mySQL equivalent in nothing flat.  I hit a huge pothole when I get to the mongo side.

The method is defined as a protected abstract method in the base class – so this method has to appear in both child classes as defined in the parent:

protected abstract function loadClassById($_key, $_value);

So I have my methods defined in both the mySQL and mongoDB middle layer.  My strategy for the mongo fetch-and-return is pretty simple — once the class has been instantiated, do the following:

  1. make sure the $_key value exists in the allowed field list
  2. make sure the $_value has a value
  3. query mongodb using .findOne()
  4. store the return key->value pairs in the member array
  5. return status

That’s pretty much it.  But I run into huge problems when I get to step 3 — use the mongoDB findOne command.

The findOne method takes an array input of the key->value pair.  From the mongo command line, you’d execute something like this:

> db.session_ses.findOne({'idpro_ses' : 1})
{
 "_id" : ObjectId("4ea1af93ddc69802376b56d1"),
 "id_ses" : 1,
 "idpro_ses" : 1
}

( Just to show you that the data exists in the mongo collection…)

But, the PHP-ized version of the method is a wee bit different:

$this->collection->findOne(array(‘idpro_ses’ => 1));

All of the examples that I’ve been able to locate show using the method by invoking it using literals.  My problem is that I have the two input parameters sent to the method ($_key and $_value) and I’ve got to find a way to get the PHP version of the method call to work using variables instead of constants.  This is what didn’t work:

$this->collection->findOne(array($_key => $_value));

$this->collection->findOne(array(“‘” . $_key . “‘” => $_value));

 

$this->collection->findOne(array(“{$_key}” => $_value));

$aryData = array();
$aryData[$_key] = $_value;
$this->collection->findOne($aryData);
or
$this->collection->findOne(var_dump($aryData)); 

I thought this worked but I was wrong:

$this->collection->findOne(array(array_keys($aryData) => array_values($aryData)));

This format returned a mongo record — the problem was that it returned the first mongo record independently of any key-search criteria.

What finally worked for me was this:

            $qs = array(); // QueryStructure
            switch($this->fieldTypes[$_k]) {
                case 'int' :
                    $_v = intval($_v);
                    break;
                case 'str' :
                    $_v = strval($_v);
                    break;
                case 'float' :
                    $_v = floatval($_v);
                    break;
            }
            $qs[$_k] = $_v;
            $aryData = $this->collection->findOne($qs);

[Update]

I encountered a similar problem when trying to update records in a mongo collection — while I could update the record from the mongo command line, I did not experience the same success in trying to execute the command from within my PHP program…

$foo = $collection->find(array(‘id_geo’ => $row['id_geo']));

Consistently failed.  No exceptions were caught, and mongo’s findLastError() reported no errors in the transaction.

After several iterations of debugging and attempting various work-arounds, I stumbled upon the solution as being one of casting.  While the variable was being evaluated in the PHP array as type int, somehow this wasn’t being interpreted that way by Mongo.  Casting the variable to an integer:

$foo = $collection->find(array(‘id_geo’ => intval($row['id_geo']))); 

 generated a successful query for both the find() and my update() functions.

As I gain experience with Mongo, I expect to discover more of these little mannerisms…

Web Services with PHP & nuSoap – Part 1.1

Category : Technical
No Gravatar

Introduction

[EDIT] – This is a re-hash of a document I wrote a couple years ago.  There’s been changes to the nuSOAP library and I wanted to document the updates relative to the tutorial series.  Also, I need to fix a lot of the broken-links and code listings since I’ve changes hosting providers since this article was first written.

This article is specific to nuSOAP release 0.9.5 on 2011-01-13.  This tutorial was updated on September 22, 2011.

Once upon a time, I was tasked with developing a web-services by my boss as the integration point between our production application and Sales Force.  At this point, although I’d heard of web services … kind of … didn’t Amazon use web services for something?  Still, I’d never coded a web-services based application before.

At this point, I have to assume you are unfamiliar with the concept of web services and why you may have to create and provide a web-services offering to your client base.  Web services allow a remote client to access functionality (as defined by you, the programmer) via the standard HTTP interface (normally, port:80) to your application and database services.

Back in the day, networking services (semaphores, pipes, streams, message queues, and other forms of IPC) were custom-written and assigned/slaved to unique networking ports for accessing specific service daemons.  Of course, the internet was a kinder, gentler place back then… and a given server may have had dozens, or even hundreds, of non-standard ports open, listening, waiting, for networking service requests.  Or hacking attempts.

Today, web-services is a replacement to dedicated networking apps – all handled by your web server, and all serviced over the same network port: port 80.  Since port 80 is a standard port and, usually, already open on a web-server, additional security risks by opening new non-standard ports for networking services are averted.  Your web-server, such as Apache, now has the responsibility of processing the request and delivering the results to the client.

The web-services component piece is a collection of functions that you’ve written that provide remote clients access to your system.  These functions are accessible only via the web services framework and while they may be duplicated from a dedicated and traditional web-based application, the web-services framework is designed as a stand-alone piece of software.  Think of the web-services piece as your application’s data-processing layer minus the presentation layer.  The “M” and “C” of the “MVC” model.

Initially, when I was tasked with a similar set of objectives, I initially tried xml-rpc.  This led me down a rabbit-hole that spanned nearly a week of my time with the end-result being abandonment.  I hit road-blocks with xml-rpc over server authentication and passing complex objects.  Exacerbating the issue overall is that xml-rpc seems to be dated technology – I had a hard time locating resources that were recent.

Then I stumbled across nuSOAP – it’s free via sourceForge, stable and, using a quote from sourceForge:

“NuSOAP is a rewrite of SOAPx4, provided by NuSphere and Dietrich Ayala. It is a set of PHP classes – no PHP extensions required – that allow developers to create and consume web services based on SOAP 1.1, WSDL 1.1 and HTTP 1.0/1.1.”

nuSOAP seemed to have more of everything available: tutorials, examples, articles, blog posts.  When I started my implementation with nuSOAP, the first thing I received help with was server-level authentication.  I was able to immediately get my remote requests validated by the web-server and handed off to the web-services module!

The major selling point, for me, on nuSOAP is that nuSOAP is self-documenting.  As part of the API functionality, nuSOAP generates HTML pages that documents the exposed services via the WSDL  and also provides you with a complete XSLT definition file!

First off, download and install the nuSOAP libraries – I provided a link to the sourceForge site a couple paragraphs ago – and unpack the tarball.  You’ll end-up with a directory (mine is called: ./nuSOAP) and, within that directory, is the one file you include: nusoap.php.

There are two pieces to this tutorial — a server side piece and a client-side piece.  While you can execute both pieces off the same environment (machine), normally you’d use the client remotely to access the API server-side code.

What’s Not Covered:

Apache.  Apache configuration for your vhost.  Apache .htaccess.  This article assumes that you’ve a working server and that you’re able to install and access the server files via Apache.  Even if your Apache server is a localhost configuration, access the server-side files via localhost client, traversing the TCP stack locally, is still a valid method for testing your web-services server application.

 

Time to push up our sleeves and start working on the server code…

The Web-Services Server

Today, we’re going to write a ping server — where the server has an exposed service (method) named “ping” which takes a single argument (a string) and returns an array back to the calling client.  The return array contains two associative members: a boolean (which should always be true – otherwise there are other issues…) and a string which is the modification of the original string in order to prove that, yes, we went there and we came back.

Because my project, and I’m doing this project for my new company, is going to represent significant effort, size and complexity, I’ve broken out the components of nuSOAP request into exterior files because, later, these will become control file which will, in turn, load files that have been organized into a hierarchy friendly to the application’s data model.

So, if this file, which I’ve named index.php, seems small, remember that you’re not viewing the dependent files (yet).

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
<pre><?php
/**
 *
 */


// setup...
require('./nuSOAP/nusoap.php');
// set namespace and initiate soap_server instance
$namespace = "http://myapi/index.php";
$server = new soap_server('');
// name the api
$server->configureWSDL("myapi");
// assign namespace
$server->wsdl->schemaTargetNamespace = $namespace;
// register services
require('myServiceRegistrations.php');
// load WSDL...
include('mywsdl.php');
// load services code modules...
require('myServicesModules.php');
// create HHTP listener:
$request = isset($HTTP_RAW_POST_DATA) ? $HTTP_RAW_POST_DATA : '';
$server->service($request);
exit();
?></pre>

So far, so good – let’s take a look at what we’ve just done:

  • we’ve included the nu_soap library…
  • declared our namespace (which is the URL of the api server)
  • instantiated a new soap_server instance and assigned it to the variable $server
  • initialized the WSDL
  • assigned the namespace variable to the WSDL
  • load and register our exposed services
  • load the WSDL
  • load the service code
  • create the HTTP listener
  • invoke the requested service
  • exit

This (index.php) file is the server-side file that will be invoked for ALL future API calls to the service.  It invokes three control files which, in turn loads the services (WSDL definitions), the WSDL variable definitions (think of these as inputs and outputs to your exposed services), and the actual code for all of the services, and their supporting functions, that you’re going to expose via your API.

Side Note:  This is the file you’ll reference in Apache when you’re (preferably) creating a new Virtual Host for the API.  HTTP requests that resolve to your server will be serviced by Apache which will, in turn, serve the results of this program back to the client.

Next, we’re going to define the myServiceRegistrations.php file that is required by index.php.  This file contains the WSDL for each and every exposed service that the API serves.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
<pre><?php
/**
 *
 *
 */


$server->register('ping', array('testString' => 'xsd:string'),
                          array('return' => 'tns:aryReturn'),
                  $namespace, false, 'rpc', 'encoded',
'<p><strong>Summary</strong>: returns response to a ping request from the client.  Response
includes testString submitted.  Used to test Server response/connectivity.
</p>
<p>
<strong>Input</strong>: $testString (type: string) and random collection suffices.
</p>
<p>
<strong>Response</strong>: Service returns an array named $aryReturn with two associative
members: &quot;status&quot; and &quot;data&quot;.<br />$aryReturn[&quot;status&quot;] should
<i>always</i> return true.  (A time-out is an implicit false.)<br />If no value was passed
to the service via $testString, then the value of $aryReturn[&quot;data&quot;] will be
empty.<br />Otherwise it will contain the string: &quot;Rcvd: {yourString} (count)&quot; to
show that the message was received and returned. (count) is a character count of the
passed string, also validating that the passed data was received and processed.
</p>
'
);</pre>

This PHP code registers a function called “ping” with the nuSOAP $server instance.  The second parameter is the input to function.  Note that all input (and output) parameters have to be declared as an array even if there’s only a single value being passed.  Also notice that you have to type-cast the variable being passed using XML datatypes.  For your data definitions, you use one of the 44 built-in datatypes defined in this document: http://www.w3.org/TR/2001/REC-xmlschema-2-20010502/.

(For more information on XSD object and XML schema, please visit: http://ws.apache.org/axis/cpp/arch/XSD_Objects.html.)

The third argument to the method “register” is the data type returned.  This is a complex variable called “aryReturn” — don’t worry about right now, we’re going to define this complex-type variable in another configuration file.

The next four arguments are:

  • our $namespace variable we set in index.php
  • boolean false
  • ‘rpc’ for the call type
  • ‘encoded’
Use these values literally.
The last variable is a huge block of HTML.  This block of HTML can be as large, or as small, as you need it to be.  It’s the basis for the WSDL documentation that nuSOAP generates for your client-side developers.
When developers hit the server URL, they’ll be presented with your API documentation that nuSOAP generates from the WSDL file(s).  As your API grows, exposed methods will be listed in the blue-ish box on the left side of the screen.  Clicking on any of the methods will expose the details (requirements) about that method thus:

Nice, huh?

The second file (that we’ve included in our source: (mywsdl.php)) is the WSDL file that defines our data structures that are used as either inputs, outputs, or both, to the exposed services.  Another thing I like about SOAP and nuSOAP is that it introduces a layer of strong-typing to the PHP stack.  You can save a bit of debugging time by requiring that you call a method with EXACTLY this and return EXACTLY that.  Anything else tends to generate a system-level error:

1
2
3
4
<pre>[Thu Sep 22 14:36:59 2011] [error] [client ::1] PHP Fatal error:  Call to a member function addComplexType() on a non-object in /htdocs/LL2/trunk/services/mywsdl.php on line 9
[Thu Sep 22 14:36:59 2011] [error] [client ::1] PHP Stack trace:
[Thu Sep 22 14:36:59 2011] [error] [client ::1] PHP   1. {main}() /htdocs/LL2/trunk/services/ll2wsdl.php:0</pre>
<pre>

This error message, from the apache error log, is somewhat obfuscated in it's meaning.   I attempted to return only the string, by itself, instead of returning the array (of two elements) that I had told nuSOAP I would return for this service.  This error was generated because the types (between the code and the WSDL) of the return variable (structure) did not exactly match.

If you've only ever coded in a loosely-typed language, like PHP, than this part of SOAP is going to be a bit of a ... transition ... for you.  When we say that something, be it a variable, function, or exposed service, is strongly typed, we're declaring the type of that object and, if the type of the object during run-time does not match, then SOAP will force PHP to throw a fatal as shown in the error log snippet above.

Keep this in-mind as you develop exposed services that are increasingly complex.  Since the error messages tend to point you at your code, at the point of failure, it's easy to forget that that the requirements of the underpinnings (in this case, the WSDL), are the root cause of your PHP fatals.

That being said, let's take a look at the WSDL for our ping service:

1
2
3
4
5
6
7
8
9
10
11
<pre><?php
/**
 * WDSL control structures
 *
 * initially, while in dev, this will be one large file but, later, as the product
 * matures, the plan will be to break out the WSDL files into associative files based
 * on the object class being defined.
 */

$server->wsdl->addComplexType('aryReturn', 'complexType', 'struct', 'all', '',
            array('status' => array('name' => 'status', 'type' => 'xsd:boolean'),
                  'data'   => array('name' => 'data',   'type' => 'xsd:string')));</pre>

We're invoking the nuSOAP method addComplexType to define a structure to the WSDL in our Table Name Space (tns).   To do this, we first define the name of the structure that we're going to use: aryReturn and then we define the composition of that structure.

The declaration for this looks a lot like a standard PHP declaration for an array with the exception of the XSD (XML Schema Definition) appended at the end of each element's declaration.  (See the links I embedded above for explanations and examples of valid XSD.)

XSD provides part of the strongly-typed concept for our structure elements.  We're telling nuSOAP to expect a variable structure containing these named elements of this type.

What we have, then, is an associative array with two elements: 'status' and 'data'.  $aryReturn['status'] and $aryReturn['data'] and they're of type BOOL and STRING respectively.

Note, finally, that this variable structure isn't confined to single-use.  Once we've declared it within our tns, it's available to any exposed service where it's needed.  This is the model for my common error structure -- the boolean indicates success or fail on some service operation and the data component contains the relative diagnostic information.

The third and final file we're including into the server source is the code for the exposed service.  This is where you write the function handlers for your services.  Since you've already defined the input parameters, and the return types for the ping service, there's very little left to do.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
<pre><?php
/**
 * ping
 *
 * service that confirms server availability
 *
 * input: any string
 *
 * output: reformatted string:  "Rcvd: {string} (charcount{string})"
 *
 * @param string $testString
 * @return string
 */

function ping($testString = '') {
    return(array('status' => true, 'data' => 'Rcvd: ' . $testString . '(' . strlen($testString) . ')'));
}</pre>

Note the following in the code for our exposed service:

  • our exposed method is named ping because that's how we registered the service (myServiceRegistrations.php)
  • we're providing a default type-cast for the input param of string in case the service is invoked without input params.
  • we're returning BOOL true and prepending "Rcvd: " to the received string, and appending a character count to prove that the service successfully responded to the client's request.
  • the return structure exactly matches the WSDL declaration: the name of the array elements, and the element types.
If you've correctly installed and referenced (within your PHP) the nuSOAP libraries, then you should be able to load the url of the new server source file into your web browser to see the nuSOAP-generated documentation for your new web services.  Click on the WSDL service function: ping to see a detailed description of the function.

If you're using IE, then clicking on the WSDL link will return the XML.  If you're using Firefox, Chrome or other browsers, clicking on WSDL will display the generated XML for your service.

Now that the server is working on it's own, it remains fairly useless until we can get a client to connect to it invoke it's methods.  Let's work on the client next...

The Web-Services Client

The web services client application will also be written in PHP.

The web client is an application that connects to remote server using the http port 80.  To do so, you'll need the client to be aware of certain bits of information that may, or may not, be required to access the remote server.

In our client, we're not going to require remote authentication -- but I'll take a quick aside andexplain how you would include this, client-side, if your server required .htaccess authentication.

nuSOAP has a client method called setCredentials which allows you to specify your .htaccess username and password and the authentication schema.  It's a single line of code which is normally used to require not only clients to login to access your API, but, once identified, you can limit the set of exposed methods available to individual clients or groups.

For example, if you have a product you've developed in-house, then you'd want full-access for your front-end web/applications servers.  Your PM later decides to open a subset of the API to the general public and a subscription-based set of exposed methods in order to monetize your product.  Finally, the PM also wants to "white-box" the product so that other companies can use it but with their branding and access to isolated or discrete data sets.

What you'd end-up with is several levels of client access to your web services.  Implementation of limiting exposed services would be handled server-side but it would be based on your client's authentication and possibly the subject of a future tutorial...

So, to the client-side code: (name this file: apiTestClient.php)

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
<pre><?php
// Pull in the NuSOAP code
require_once('./nuSOAP/nusoap.php');
$proxyhost = isset($_POST['proxyhost']) ? $_POST['proxyhost'] : '';
$proxyport = isset($_POST['proxyport']) ? $_POST['proxyport'] : '';
$proxyusername = isset($_POST['proxyusername']) ? $_POST['proxyusername'] : '';
$proxypassword = isset($_POST['proxypassword']) ? $_POST['proxypassword'] : '';
$useCURL = isset($_POST['usecurl']) ? $_POST['usecurl'] : '0';
$client = new nusoap_client('http://{YOURSERVERURLHERE}/index.php', false, $proxyhost, $proxyport, $proxyusername, $proxypassword);
$err = $client->getError();
if ($err) {
    echo '<h2>Constructor error</h2><pre>' . $err . '</pre>';
}
$client->setUseCurl($useCURL);
$client->useHTTPPersistentConnection();
// Call the SOAP method
$result = $client->call('ping', array('testString' => 'Argle'), 'http://localhost');
// Display the result
if (!$result) {
    echo "service returned false";
} else {
    print_r($result);
}
unset($client);</pre>
<pre>?></pre>

The first thing we do in our client-side code is to include the nuSOAP libraries.

The next five lines of code read from the local POST environment, testing if you've established a proxy for your web-services server and, if so, populating the proxy variables.

The next line instantiates a nuSOAP client and associates it with our remote server.  Change "YOURSERVERURLHERE" to the name of your apache web server URL where you have the server side code installed.  (e.g.: localhost, myserver.com, etc.)

Note the name of the function call: newsoap_client()...as opposed to using the function soapclient().  This function name is legacy-compatible with PHP 5.0's instantiation call:  new soapclient() - the PHP SOAP extension uses the same instantiation function name as the nuSOAP library.  If you have both installed, (PHP 5.0 SOAP extension, and the nuSOAP libraries), executing the client will return errors as you've overloaded the soapclient() function.  (You're calling the PHP SOAP function with the nuSOAP function parameters.)  Rename the soapclient() function to the back-compatible function: newsoap_client().

The next two lines tell the nuSOAP client to useCurl, when possible and if previously saved and, if possible, to use HTTP persistent connections.

Next, we're going to consume the web-services from the server by making a call to the nuSOAP method: call().  The arguments to this method are:

  1. the name of the service being consumed (ping)
  2. the input string (note:  all inputs must be passed as arrays!)
  3. the namespace URI (optional:  WSDL can override)

Store the results of the web-services to the aptly-named variable $results and evaluate it upon return.  If the client call was not successful, then display an error message.

If the client call was successful, then display the contents of the returned array.

Note that you can run this client code from either a browser using the "file://" option or, if the client source code is accessible to apache, then you can display using a browser.

When I run this client-side software in the browser, I get displayed back:

Array ( [status] => 1 [data] => Rcvd: Argle(5) )

So, how do we know that the client went out and successfully returned from the server with the data?  Simple - in the client source, you will not find the literal "Rcvd:" anywhere in the source.  This data was supplied by the server-side function and returned back to the client.  It's a simple example, but it provides proof-of-concept that we're successfully able to connect to a remote web-server and return data created on the remote server back to the client program.

Let's wrap this up...

Summary:

This tutorial (hopefully) explained what web-services are, and provided you with a practical example of a consumable service: ping().  Such a service would normally be invoked as a means of testing server availability.

We created a web-services server file using the nuSOAP library by defining a complex-structure (an associative array) and registering a method with the nuSOAP server.  The method takes an input parameter which, although it's only a single input parameter, must be built and passed as an array construct to the server method.  Next, we showed that by loading the server source into a browser, we learned the nuSOAP provides built-in documentation for your web-services structures and methods.  Which is a very nice-to-have when you're creating your developer documentation!  Next we created the web-services client source code file which connected to our remote server and invoked the server's method: hellow().  If all worked correctly, you displayed the string returned from the remote server within your browser window.

In the next installment, I'll cover the web-services server-side of this business and we'll see how to create the various methods for accessing a mySQL database, complex structures as input and output parameters to those methods, and general debugging techniques.

Thank-you for your patience - I hope this article helped you.

September 26, 2011

Waaaaake-up! Hello? Lion? You awake? WAKE-UP!

Category : Rant, WTF
No Gravatar

Oh, Apple.  What did you do now?

It’s one thing to introduce broken (or bent) functionality in an upgrade release.  It’s quite another to break (or bend) existing functionality in the same upgrade.

I really like Lion so far.  What I thought I would miss, I don’t, and I’ve already become dependent on several of the base features that the upgrade offers.

And, hey, Microsoft (you big wad o’ suck) take note:  a major update for $30 that I can install on all of my machines!  And I don’t have to pay attention to see if it’s ultimate home premium 64, too!

(aside:  I’m more pissed that usual at Microsuck.  Earlier, using Bootcamp, I was playing Rift and I noticed that performance was lagging badly.  To the point where I just decided to log-out and get some work done.  After logging, I see that my tx/rx light on the dsl modem is solid.  During shut-down, I see the usual dire-imprecations and deadly warning spew that pops when you update a Microsuck in-progress system update download.

WTF?  I explicitly turned off the “feature” of independent updates in favor of only-update-when-I-tell-you option.  You know, the way real operating systems do it.  Apparently this setting means jack-shit as the crapware decided, again and on it’s own, to go out and download god knows what from the ‘net.  Pure and unadulterated hubris.

Now I don’t mind the constant virus updates — I deleted three security exceptions from the Windows box today alone.  But this constant updating without my permission really is pushing it.  You confirm everything I want to do, concerning downloaded content, several times.  But true to the “do as I say not as I do” philosophy of this bloatware, Windows continues to ignore user selections and configurations and just farts and whistles it’s way through a continuous stream of critical updates.  Pure crapware.

Wanna end the wars in Iraq and Afghanistan?  Send them free copies of Windows to install on all their military infrastructure.  War will be over in a week, guaranteed.)

That was a long “aside”.  Or rant.  Or some factual observations.  Whatever.

Anyway, back to a real operating system that not only let’s you get real-work done, but also listens, remembers, and then doesn’t ignore your configuration settings…

I’ve been having problems with my Lion installation not waking from deep sleep.  I define two levels of sleep.  One is light-sleep: where the computer’s screen saver kicks-in, and a simple mouse-twitch brings it back.  The other is deep sleep: this would be when you explicitly put the computer to sleep, or your power management settings kick in.

What I’ve been experiencing has been happening either on weekend-mornings, or in the evenings when I get home from work.  I sit down at the computer and poke the shift key, twitch the mouse, tap the space bar and … nothing.  Repeat shit-key poke, mouse twitching, space bar tapping. … Still nothing.

I poke the caps-lock key.  … No light.  This is not good.

Both my keyboard and my mouse are wired USB peripherals.  So I dis(re)connect the devices from the hub and, again, twitch the mouse, poke the cap-lock key and … black screen.  There is no power indicator on the new 27″ iMacs.  So I have no idea what state the computer thinks it’s in.  Time for some drastics.

I tap the power button.  This is usually enough, on my MacBook Pro, to jog it awake but, on my iMac…nothing.

Eventually, frustration wins out and I do a hard-reset by holding down the power key until it powers off and then I reboot.

Goddamnit.

I have a three support contract with Apple on this desktop but I’ll be damned if I’m going to call them to confess that I’ve no idea on how to wake-up my desktop from sleep.  So, I google it.

I found this article, which explains how to reset the PRAM and NVRAM on your iMac because, you know, batteries get old and flash memory gets stupid over time.  So I follow the steps and, when the computer restarts, it’s definitely brighter.  (I’m not that good of a touch typist and I tend to inadvertently do things to both the brightness and volume controls…)

But, the next day when I get home from work, the computer is back in Rainman mode and I have to power-down to bring it back.

So I google it again, and this time I see a post on a mac-forum that blames the problem on disk permissions.  Sure.  Why not?  So I run verify disk and, lo’!  I have a bunch of crap that gets re-perm’d.

Still not going to call Apple.

I’m writing this article and I guess I’ll see what happens the next time I try to roust the machine from deep-sleep.  I’m pretty confident that it’s going to fail and, if it does, then I’ll log a call to tech support.

In the meantime, if any of you have suggestions, I’m open…

 

 

X11Forwarding from CentOS 6 Linux to Mac OS X Lion via SSH

Category : Technical
No Gravatar

In my previous post, I wrote about getting gpass (a password manager for the gnome desktop) compiled from source and running on our CentOS 6 platform.  The screenie I took of the welcome screen was a mac-i-fied version.

I had configured my Linux machine to support X11 port-forwarding over a secure shell.  It was surprisingly quick and easy to set-up and execute.

I wanted to remote-display the gpass window to my Mac OS X Lion desktop because I needed to transfer passwords from my 1Password application (running on Lion) to my gpass (Linux) program.  Some of the passwords are pretty gnarly so the only way I can guarantee transferring data without making typos was to set-up a copy-paste-friendly environment.

One quick caveat. I’ve noticed that, when I terminate an X11 program from my Lion shell, I can no longer use that shell to initialize another X11 applet.  I need to exit and re-start the terminal.  If you know of the work-around for this, please leave a comment/reply to this post.

For all the following commands, it is assumed you have sudo privileges on your Linux system.

The first step I took was to edit the /etc/ssh/ssh_config file.  At the end of the file, past the comments, there is a section labeled:

Host *

ForwardX11Trusted yes
X11 Forwarding yes

Make sure that you have those two lines, uncommented and present, in your configuration.

Next, (re)start your sshd server:

# /etc/init.d/sshd restart

Stopping sshd:                                         [ FAILED ]
Generating SSH1 RSA host key:         [      OK      ]
Generating SSH2 RSA host key:         [      OK      ]
Generating SSH2 DSA host key:         [      OK      ]
Starting sshd:                                           [      OK      ]

 

In case you’re curious, the FAILED message in the first line of output was generated because I didn’t already have sshd running on my system.

My machines run on a 192.168 subnet behind two firewalls – the firewall on my DSL modem, and the firewall on my multi-port router.  Normally, I’m not too concerned about the security of my individual machines.  (e.g.: I’m not running a software firewall on my Mac or my Linux server.)  My subnet is DHCP-served by my router and the router is on it’s own subnet DHCP-served by the dsl router/modem.

I need to obtain the current IP address of my linux server which I do so my running the ipconfig command.

Next, I switch over to my Mac and open a terminal — within the terminal, I enter:

iMac:~ mike$ ssh -X 192.168.0.6
The authenticity of host '192.168.0.6 (192.168.0.6)' can't be established.
RSA key fingerprint is f9:04:2d:0e:70:3d:a7:8f:92:c0:02:69:8c:f2:e6:51.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added '192.168.0.6' (RSA) to the list of known hosts.
mike@192.168.0.6's password:
whassup?
/usr/bin/xauth: creating new authority file /home/mike/.Xauthority
[mike@codeMonkey ~]$

At the command prompt, I now only have to enter whatever X11 command and that program will be displayed on my Mac Desktop.  I can even open and start an entire desktop session.  I could – but I won’t — my Linux server only has 2gB of Ram…

Instead, I’ll open a gnome-terminal.  So, at the prompt, I simply type: gnome-terminal and I get the gnome-terminal to appear on my desktop:

That’s pretty much all there is to it, as far as I could tell.  Eazy-peezy.

One last note — once you have a terminal running on your Lion desktop, then any X11 commands, such as gpass, you enter will all be displayed on your Lion desktop.  This circumvents the one-terminal-one-applet restriction I mentioned at the top of this article.

That’s pretty much it for this article — hope this helps!

Installing gpass on CentOS 6 Linux

Category : Technical
No Gravatar


Over the last year I have become utterly dependent on a product called 1Password by Agile Bits software.  For those of you that are unfamiliar with this software, 1Password is a multi-platform program that manages all your passwords, in additional to other sensitive information, in an easy-to-use interface.

Originally written for the Mac, the software is now offered on iPad, iPod, ‘droid, and Windows machines.  I have it installed on all available platforms.  While initially bemoaning the cost of the product – it’s not cheap – I’ve come to depend on it for all of password storage, my software license management, and even the credit-card information for the card I use for online purchases and subscriptions.

Quick aside and then I’ll cease the fanboi gushing: my favorite feature of the program is the password generator.  I can custom-tailor a password to be as obnoxiously long, and obfuscated, as I need and I don’t ever, ever, have to type it in when challenged.  Passwords are simply copy-pasted from the 1Password program, or you can use the embedded 1-click feature functionality of the support extensions available for all browsers.

My only complaint with 1Password is the lack of Linux support.  Since I’m using Linux as my LAMP development platform while at-home, I need a comparable password manager. I know I won’t have all of the slick features of 1Password, but at least I’ll be able to copy-paste long, obfuscated, passwords from the password manager into my Linux desktop applications.

So, let’s get started!

There’s some good tutorials already available on the ‘net about doing just this – however, none I found were exactly right and, following those tutorials, I did run into several side issues.  I’ll cover all those issues here so that your installation will be seamless.

Operating System: CentOS 6 Linux
Desktop GUI: Gnome
gPass version: 0.5.1
EPEL repository: 6.5

Download the gpass source into your “Downloads” directory and unpack the tarball:

wget http://projects.netlab.jp/gpass/release/gpass-0.5.1.tar.gz

tar xvzf gpass-0.5.1.tar.gz

cd gpass-0.5.1

I based my initial install of gpass from the UnixCraft blog post here.  (In the tutorial, they omitted the arguments to the tar command to un-tar the tarball that creates the gpass source directory.)

In step 1, the blog asks you to do a group install of the development tools and, secondly, install the gnome-ui, mhash, and mcrypt development libraries.  The second step failed for me following the successful install of the gnome-ui as my stock yum configuration was unable to locate either the mhash or the mcrypt packages.

After googling the issue, I determined that I needed to at the EPEL repository to my yum configuration.  It’s common to have several repositories in your yum catalog.  You’ll add additional repositories by establishing configuration files in /etc/yum.repos.d/.

Setting up the EPEL repository is pretty easy as they’ve created an rpm just for this purpose.  Make sure you have sudo privileges on your account and enter the following commands: (I’m currently in the “Downloads” directory in my $HOME.)

wget http://download.fedora.redhat.com/pub/epel/6/x86_64/epel-release-6-5.noarch.rpm
...
rpm -Uvh epel-release-6-5.noarch.rpm

Side note: I’m aware when I’m reading how-to’s on other sites that reference software versions that said versions may not always be the current, and most stable, release available today.  I always check the repository, using a browser, before downloading to ensure I’m obtaining the latest version.

Once the rpm is installed, you’ll need to edit the repository file.  Again, using sudo, edit the /etc/yum.repos.d/epel.repo file and in the EPEL repository section, add the line: priority=3 at the end of the section.

I’m now ready to install the mhash and mcrypt packages, obtaining them from the Redhat EPEL repository.  Again, assuming sudo privileges:

# yum install libmcrypt-devel
# yum install mhash-devel

From this point, you need merely to follow the instructions in the UnixCraft blog I linked-to above, but here are the steps to finish the installation.  Again, assuming you’ve changed-directory to the gpass source:

./configure

./make

./make install

At this point, as long as you’ve not seen any error messages in your output, your gpass program is ready to use.  Test by typing gpass at the command line — you should see the gpass window pop-up on your desktop:

In the screen-shot to the right, those of you that are past your second cup of coffee may have noticed that my gpass window looks suspiciously like a Mac OS X version.

I am running the gpass application on my Linux server, but I am serving the display to Mac OS X Lion desktop.  I set-up the configuration to do this for two reasons.

  1. to capture and display screenies
  2. to copy paste data from my native Mac 1Password application into my Linux gpass application.  I do NOT want to retype some of those passwords…
That’s pretty much it.  I leave the exploration and use of gpass up to you.  I’ll do a follow-up tutorial quick-post on how-to set-up XForwarding on Linux to your remote desktop (Mac) via secure shell.
Thanks for reading – hope this helps!

 

Part 5: Setting-up a Linux Development Machine: Virtual Hosts in Apache2

Category : Technical
No Gravatar

When I am working on code project, I isolate that project within it’s own directory/repository.  Further, it matters not if I’m starting a completely new project, or if I’m branching off the trunk of an existing project.  As a means of imposing order over chaos, I isolate the existing project within it’s own sandbox both on the filesystem and via Apache2.

To do so requires an understanding, somewhat, of the mechanics of Apache2, DNS, and your localhost.  A minimal understanding, trust me.

What it, in return, gives you is an isolated view of your code project from the web-server perspective.  Cookies are isolated by domain, your document root is isolated to a single directory/repository, and you not only put your log files, just for that domain, where ever you want but you can also name them anything you want as well.

What I’ll provide you with in this installment is a rudimentary understanding of the mechanics behind virtual hosting using Apache2, a template configuration file to get you going, and the basic steps necessary to get the whole mess working.  Let’s get started…

When you start a new project, if you’re checking it out from a source-code repository, you’ll typically assign it to a directory somewhere common.  For example, within your home directory, you may have a folder named “code” and beneath that folder, other folders that describe either the project or the programming language you’re working in.  Doesn’t really matter as the point is this:  you’ve isolated your code repository from everything else on your filesystem, right?

It really doesn’t matter, to Apache2, where you create your filesystem repository.  As long as the webserver pseudo-user has access permissions to the directory, then you can access the files within that directory via a web browser.  The webserver has to be configured to be told that, for a given domain name, where is the documentRoot for that domain.

Some of you, at this point, may be asking: what’s a domain name and why is it important?  Well, a domain name is simply a name you’ve assigned to the project to keep it separate, at least in your own head, from the other projects you may, or may not, have running on your development machine.  For example, I create a new project called newWidget and it’s currently in the 1.4 revision.  I’m ready to branch and write some new features for the product so, using whatever sccs tool, I branch off the trunk and create the 1.5 branch.

I check that branch out to a directory in /lampdev/php/newWidget115.  I now need to do two basic things:

  1. invent some domain name that will be used exclusively for this project and resolve the domain to my localhost
  2. create a virtual host in apache so that apache knows that http://newW115 points to my localhost

The reasons, apart from what we’ve already discussed, is to keep your local DNS services on your local machine.  If you, before entering any configuration information, entered: http://newW115 into a browser url bar, chances are very good you’re going to end-up on a search page (I’m using Chrome) or get some sort of browser error.

So the first step is to define the new domain name (again, given that we’re already checked the code out into the aforementioned directory) to the local system so that all requests to that domain are resolved locally through our name services.  To do this, we’re going to sudo edit the /etc/hosts file.

This file, /etc/hosts, is the first thing checked whenever your local name services is trying to resolve a host name.  If it finds a host-to-IP alias in this file, all further attempts at resolution are halted as it has successfully resolved the host name.  Edit /etc/hosts to resolve your new domain.  It should look something like this:

127.0.0.1    localhost codemonkey codemonkey.shallop.com codeMonkey.shallop.com newW115

The way /etc/hosts works is that you first list an IP address for the domain to resolve to - in this case, we’re using 127.0.0.1 which is TCP/IP speak for your local host.  Next we list all of the domain names that are going to resolve to this IP address.  In the example above, we’re resolving localhost, codemonkey, codemonkey.shallop.com, codeMonkey.shallop.com, and the new domain: newW115 all to 127.0.0.1.

Whenever I type one of these domains, for example, in to a web browser URL bar, my local host domain services won’t go out to my network name servers to resolve the domain name — it’s telling the requesting service that it’s 127.0.0.1.  Note, too, that you can alias multiple domain names to the same machine.

Side Note — this is how you can blacklist certain domains from your browsing experience.  Simple resolve that domain to 127.0.0.1…but that’s an article for another day…

You can also have multiple entries resolving to the same IP address.  It would have been just as correct for me to have listed by /etc/hosts file as:

127.0.0.1     localhost
127.0.0.1     codemonkey
127.0.0.1     codeMonkey
127.0.0.1     codemonkey.shallop.com
127.0.0.1     codeMonkey.shallop.com
127.0.0.1     newW115

Finally, also note that a domain extension isn’t really required.  We can name our domain pretty much anything we want and as long as you universally use that spelling (and case), then it will resolve locally.

Now that the domain is resolving locally, the next step is to tell Apache2 how to handle the request.  When you type: http://newW115 at the browser, the browser will query local services and receive a response that the domain is handled locally.  Apache2 will then say: “Oh, if it’s local, then were do I go to get the files and stuff?”

The configuration for Apache2 is done with virtual hosting.  Technically, you can do this without virtual hosting — but you can only do it for one domain.  If you want to locally-host multiple domains, you have to use virtual hosting.

The Apache2 configuration file lives in: /etc/httpd/conf and is named: httpd.conf.  This is the main configuration file for Apache2.  Some installations use a sub-directory, usually called something like: vhostsd.conf, and stores the vhosts.conf file within that directory.  That’s ok, too.  Apache2 is versatile that way but, for our purposes, we’re going to maintain the virtual host configuration(s) within the main conf file.

However, if you wanted to use a separate file for Virtual Hosting, all you need in your httpd.conf file is the directive:

# Virtual hosts
Include conf/extra/httpd-vhosts.conf

At the very end of httpd.conf, there’s a section called: Name-Based Virtual hosting.  We’re going to append this virtual host configuration to the end of this file.

Allow me to side-step for a quick second.  Consider if we were to install phpMyAdmin locally on our server because this is how we want to administer our mySQL database.  We can install the program files anywhere as phpMyAdmin is just another LAMP application, right?  Were we to do that, then we would need a <Directory> directive to Apache2 telling Apache2 where to look for phpMyAdmin.  The domain for phpMyAdmin would still be localhost, or 127.0.0.1 or whatever else you’d defined in /etc/hosts.  The location of the application can live anywhere and we’re using the conf file to tell Apache2 how to find and serve it to us when requested.

Alias /phpMyAdmin "/opt/local/www/phpmyadmin"
&lt;Directory "/opt/local/www/phpmyadmin"&gt;
Options Indexes MultiViews
AllowOverride None
Order allow,deny
Allow from all
&lt;/Directory&gt;

What this <Directory> directive simply does is tell Apache2 where to look for phpMyAdmin if I enter something like: http://localhost/phpMyAdmin in the URL bar of my browser.  It’s not the same thing as giving phpMyAdmin it’s own domain at all.

I do this with a lot of my web applications: phpMyAdmin, mcmon, ajaxmytop, nagios, etc., simply because I don’t want to remember where the fill path name is of the applications.  It’s easier to type: http://localhost/phpMyAdmin that it is to type: http://localhost/webapps/database/phpMyAdmin.

Ok, so back to domains.  Here’s the template for the virtual host we’ve created in /etc/hosts: newW115:

&lt;VirtualHost *:80&gt;
ServerName  <strong>newW115</strong>
ServerAdmin <a href="mailto:mshallop@nileguide.com">mshallop@g</a>mail.com
DocumentRoot <strong>/code/webapps/LAMP/newWidget/1-15</strong>

DirectoryIndex  index.php

&lt;Directory /&gt;
Options FollowSymLinks
AllowOverride None
&lt;/Directory&gt;
&lt;Directory <strong>/code/webapps/LAMP/newWidget/1-15</strong>&gt;
Options Indexes FollowSymLinks MultiViews
AllowOverride All
Order allow,deny
allow from all
&lt;/Directory&gt;

ScriptAlias /cgi-bin/ /usr/lib/cgi-bin/
&lt;Directory "/usr/lib/cgi-bin"&gt;
AllowOverride None
Options +ExecCGI -MultiViews +SymLinksIfOwnerMatch
Order allow,deny
Allow from all
&lt;/Directory&gt;

<strong>ErrorLog /var/logs/115_error.</strong><wbr><strong>log</strong>

LogFormat       "%h %l %u %t \"%r\" %&gt;s %b \"%{Referer}i\" \"%{User-Agent}i\"" combined
LogFormat       "%h %l %u %t \"%r\" %&gt;s %b" common
LogFormat       "%{Referer}i -&gt; %U" referer
LogFormat       "%{User-agent}i" agent
<strong>       CustomLog       /var/logs/115_log common</strong>
<strong>       ErrorLog        /var/logs/115_error_</strong><wbr><strong>log</strong>

# Possible values include: debug, info, notice, warn, error, crit,
# alert, emerg.
LogLevel warn

CustomLog /var/logs/115_access.<wbr>log combined
ServerSignature On

&lt;/VirtualHost&gt;
</wbr></wbr></wbr>

This is a pretty minimal configuration — but it’s the boilerplate template I use for all new domains and it works.  The lines that in boldface are the lines you should change to match your environment.  Note that you can pretty much put files, such as the log files, where ever you wish.  I changed the names from my normal location but, as a rule, I maintain the entire environment outside of the root filesystem.

Once you’ve made your changes and saved the file, you’ll need to restart Apache2 so that it will read the new configuration.  If there are errors in your configuration file, Apache2 will let you know and will refuse to start.  Make sure you’ve corrected all errors and, once the server successfully restarts, you should be able to type: http://newW115 into your browser URL bar and have that domain resolve locally, and serve files from the directory you specified in the httpd.conf file.

Over time, as you add additional projects and create new code-domains, you can simply add the new <VirtualHost> directives, appending them, to the httpd.conf file as needed.  When you expire and remove hosts and files, don’t forget to remove them from the Apache configuration as well.

And that’s pretty much it.  This is a simple thing to set-up as we didn’t delve into anything that wasn’t plain-vanilla.  For example: SSL configurations, .htacces, or the re-write engine.  That’s for another day, another article.

Hope this helps…

Review: AZIO KB333BM Bluetooth Wireless Keyboard for Mac, iPad, iPhone

Category : Technical
No Gravatar

I have an iMac 27″ I7 — I wanted to try this keyboard because I needed to recover space on my desktop. I run my iMac in Windows via Bootcamp quite a bit when I’m not working/coding to play games so it was important that his kb also work under bootcamp.

When I unpacked the keyboard, I was instantly disappointed in the style and construction. It’s not quite as small as the mac wireless keyboard, measuring almost 2″ wider and about 1″ wider. It also has a cheap feel to it — there’s something rattling around in the antenna housing and the keys are a die-cut plastic. On Apple kb’s, the keys are smooth giving the kb an almost rubberized texture — they’re also solidly mounted so there’s no “play” or travel in the keys. On the Azio, the keys feel tactically different and there’s a ton of play in the keys — it’s almost like they’re mounted on swivels.

Installing the batteries was fairly easy — but the battery door is cheap, thin, plastic. It’s definitely a failure point over time. Pressing the connect button isn’t easy — the placement is on the bottom of the keyboard, along the back edge, so the button has to be recessed so you don’t tap it during normal use. There’s no tactile feedback when you do click the button to initiate a connect — you have to flip the kb over to see if the blue light has lit.

When I went to sync the kb, I was in windows 7, and attempting the sync immediately brought the computer down with the BSOD. Seeing how it was windows, it didn’t surprise me much so I re-booted into Leopard. Pairing the device didn’t work — when it asked me to type in the sequence of numbers, there was no feedback to the screen so eventually Apple asked me to identify the key to the right of the right shift key.

Which is an up-arrow. Which wasn’t recognized by Apple as a key. Which meant I had to select from a menu of choice of what type of keyboard I had. So I selected the only viable option – US/English 101 key.

I rebooted trying to get into Bootmanager — as the computer rebooted and I heard the start-up tone, I pressed and held-down the option key. The blue light on the kb flashed furiously for a second or two, then the machine booted me into Mac mode, bypassing completely the bootmanager. I re-paired the device by removing it and re-discovered. This time, without the feedback (which I realize may be an Apple issue and not an Azio issue), I just blindly typed-in the numbers without pause and the computer accepted the keyboard pairing.

Rebooting the machine, however, produced the same results as before – the kb was not recognized, not was my holding down the option key during boot, and again the Bootmanager was bypassed.

The keyboard itself feels cramped and awkward. The keys appear to be both slightly (about 1/8″) smaller than Apple’s kb, and they’re set closer-together. There additional width of the keyboard is allocated to keys along the right side, two columns, F13-F16, home, end, delete, page-up/down, and the 4-arrow keys. Totally unnecessary to add these keys and increase the form-factor imo. Even tho this is advertised as a mac kb, they couldn’t break the windows dependencies…there’s also the unnecessary function key just to screw up your typing, right under the left shift key.

I’ll try this kb out with my iPad — perhaps it will encourage me to use my iPad more for text-input. Otherwise, this device is simply garage-sale fodder. If you want a smaller keyboard, then get the keyboard here on Amazon (Super Slim USB kb) — it’s wired, but it works well. Or spend the big-bucks and try the Apple keyboard.

tl;dr: Keyboard feels cheap and loose. Could not access bootmanager. Pairing causes BSOD in windows

Part 4: Installing Apache Thrift: Linux Development Environment

Category : Technical
No Gravatar

Previously, we dealt with getting a working LAMP development environment up and running on a fresh CentOS 6 install.  We next dealt with the installation of PHPStorm and our JDK issues.

In this, and the next issue, I’m going to talk about the Thrift framework and getting it installed and running.

Thrift was originally developed by Facebook, was entered into open source in 2007, and became part of the Apache incubator the next year.

Thrift, according to Apache, is “a software framework for scalable cross-language services development. It combines a software stack with a code generation engine to build services that work efficiently and seamlessly between C++, Java, Python, PHP, Ruby, Erlang, Perl, Haskell, C#, Cocoa, JavaScript, Node.js, Smalltalk, and OCaml.”

What it is in plainspeak is an API framework for your LAMP application.

Why I want it:  I want to use Thrift for our project because of the nature of the project.  (A social-networking concept.)  Because the application will rely heavily on data-storage calls, I’ve decided to implement the data access layer as an API instead of a more-traditional OOP model.  Thrift, as the API framework, allows me complete freedom on the back-end of the API.  I can implement the API in a variety of languages, although I’ll probably use PHP.

Thrift also provides me with a strongly-typed interface to the API.  Like XML-RPC, calls to the API are well-defined beforehand and must comply with the typed definition of both the methods used, and the data exchanged to/from said methods.

My personal experience with Thrift is limited — I used it as an API for a product concept at a former employer.  The calling application would invoke the API and make requests to the API which, in turn, would do a “bunch of stuff” and return a well-defined “data ball” (a json object) back to the calling stub for processing and display.

The other concept that makes me embrace Thrift as the controller for my LAMP application is that I can completely encapsulate the data layer from the front-end developers.  They do not need to know if the data is stored within mongodb, mysql, or a flat file.  All they need is the data.  The query language is hidden; front-end developers should not need to write data-access code.

I’ll talk more about the glories of Thrift later.  For now, let’s just get it installed and running…

On our Linux system, we have to do some preliminary installation of packages first.  Luckily, if you hit the Thrift Wiki, you’ll find pretty much everything you need to do a successful install.  Be warned, however.  Sparseness of documentation could easily be one of the hallmarks of Thrift.  Read carefully, and then read again before punching the enter key on your keyboard.  Make sure you understand what it is you’re about to do.

Ok.  Let’s get some non-LAMP development tools installed.  Our first command will be to install most of the pre-requisite packages needed by Thrift:

#  <span class="Apple-style-span" style="font-family: Consolas, Monaco, monospace; font-size: 12px; line-height: 18px; white-space: pre;">sudo yum install automake libtool flex bison pkgconfig gcc-c++ boost-devel libevent-devel zlib-devel python-devel ruby-devel</span>

This  will install the base development packages you’re going to need.  Once this has completed, you should also install the open-SSL development libs as the build will fail without it.  (At least, if failed on my install.)

#  sudo yum install openssl-devel.x86_64

Installing this package will also pick-up all the dependent packages you’ll need to complete the install.

Next, download the Thrift tarball from the site and move the package somewhere within what will become your DocumentRoot path for Apache2.

#  tar xvzf thrift-0.7.0.tar.gz

Once you’ve expanded the tarball, cd into the thrift directory and follow the instructions to make the Thrift packages and libraries.  I did this pretty much exactly as told and my installation went without a problem.

At this point, we’ve only built and installed the Thrift libraries (installed in /usr/lib, I believe…).  In the next installation, we’re going to install the PHP src directory and make it visible to our application’s docRoot.

Welcome to the Real World, Rookie.

Category : Rant, WTF
No Gravatar

This is a true story…or at least this is how I remember it when it was told me about 30 years ago.

Two small-town cops walk up to the front-door of a house where they’re going to attempt to serve a misdemeanor warrant.

(The cop telling the story explained to me that a misdemeanor is a “piddly-ass” warrant – it’s usually for minor infractions and seldom requires anything more than a minimal cash bond.  This, as opposed to felony warrant for which, now-a-days, they’ll call out the SWAT team to serve it, on a Friday, where you’ll sit with a no-bond hold until sometime Monday when you appear before a judge who will set your bond.  Misdemeanor: no big deal.  Felony:  big deal.  Ok?)

One of the cops (the one telling me this story) is the “veteran” — he’s training the other cop, the “rookie” who was fresh from cop-school.  Just as in every movie, the veteran had instructed the rookie to forget everything they taught him in rookie-school because this is how it’s done on the streets.

So, as they’re approaching the front door, the veteran cop is watching the rookie cop, evaluating him for mistakes, gigs, so that he can school him on being a better cop, dig?  It’s a routine call and it’s a training mission.  The veteran tells the rookie that he’s “handling” the call — they’re to approach the house, determine if the person named on the warrant is within the house and, if so, serve the warrant.

The veteran cop hangs back as they approach the front door.  He nods in approval as the rookie approaches and takes a position at the side of the door following safety protocol, while he knocks on the door with his flashlight/beatstick.

No answer.

The rookie knocks again, louder.  Waits a few seconds then glances at the veteran.  The vet nods to the rookie.

The rookie beats the hell out of the door and yells, at the same time:  POLICE.

No answer.

The veteran is starting to relax while the rookie is getting spun-up as he *knows* someone has to be inside the house, laughing at him, laughing at his auth-or-a-tie.  He creams the door again with the light, screaming:  POLICE.  OPEN THE DOOR OR ELSE!

The veteran looks up the rookie, a smile starting to crinkle his eyes.  He says:  Or else what, rookie?

Non-plussed the rookie stands there, mouth open, no answer forthcoming.

The veteran is relentless: Or else what, rook?  It’s a misdemeanor warrant.  Or else what?

The rookie, like the proverbial deer in the headlights, just stares at the veteran.  Finally he shrugs a small  I-unno back to the veteran.

The veteran starts to laugh: Or else what, rook?  Or else we leave and go back to the station?   He shakes his head.  Come on rookie.  Let’s go; no one’s home.

I’m telling this story because the one kernel of truth in this story is the advice about “forget everything you ever learned in the academy”…  I believe it to be (a) advice common to pretty much every profession requiring training, and (b) valid in that it represents the difference between practical and theoretical.

I’m coming to the point, so bear with me please.

Theoretical is when you’re taught that the Software-Development-Life-Cycle is a process of which coding actually takes between 15-20% of the total effort.  Practical is that seldom are you given a decent spec to work off of, often time you’re cramming features into product simply because some sales dweeb,who can’t even set the gps in his beemer, told a customer “oh, yah, it does that too” when demo’ing your product and god help you if you risk his commission over something as trivial as a missing feature even if it wasn’t in what you laughingly refer to as the “spec”.

Theoretical is refining your code in iterative processes, performing code reviews with well-mannered engineers, and getting polite notices from Q/A about the latest “opportunity for improvement”.  Reality is the “if it compiles, ship-it!” work-ethic.  Spending massive amounts of time at work trying to get the software to limp out the door on it’s own without patching the patch that patched the hotfix for the patch.  Product Managers balancing staggering workloads under the euphemism of a “sprint” while you watch your family age courtesy of the updated photographs your wife emails to you every Christmas, and your boss informing you that if you don’t come in this weekend to work, perhaps a start-up environment isn’t for you.

Theoretical is getting paid vacations as down-time, off-time, time actually away from the office.  A complete and total absence of a work-presence.  Reality is being on-call 24x7x365 and, if you’re lucky enough to get a vacation approved, it’s never in a chunk of time long enough to do anything other than spend “free” time imprisoned with either in-laws or the strangers you think are your children.  To you, “vacation” is the week of Thanksgiving and the two weeks between Christmas and New Years when all the sales/marketing dweebs are suntanning in Aruba, your boss is at home gorging on eggnog and fruitcake, and the office is blessedly quiet enough to where you can almost get caught-up on your code-load because you’re not wasting half your work day in pointless meetings.

Theoretical is all your projects that you were going to develop in your “off time”, all of the open-source projects you planned on  making substantial contributions to gain international recognition,  and the agony of having to pare your resume down to three pages because of your efforts in on-going self-directed research.  Hah.  The reality is that the great side project you’ve dreamed about your entire career is abandoned on a 3 1/2″ floppy somewhere in a cardboard box in your packed-full-with-everything-but-a-car garage.

Once you leave the ivy-covered towers of academia, the world is waiting to poke you smack-dab in the eye with the reality finger.  Elegant code isn’t the priority anymore nor are esoteric algorithms, obscure architecture theorems, or theoretical models.  What’s important is that you’ve built the the product, to spec, within the allocated time frame.  Everything, and by this I mean, well, everything, else is secondary.  And, in the real world, secondary never gets bandwidth.

Because the second you’ve completed your current project, the very minute, you’re already racing to catch-up with the next project you’ve been assigned to, several weeks ago, but haven’t had the bandwidth (yet) to fully engage so, out of the gate, you’re behind.  Within a few days, your short-term memory will have purged your previous project from it’s banks, replacing it with your current crisis-code.  Maintenance developers coming to you for guidance and hints walk away frustrated because you weren’t on your last project long enough to dump the fine points into long-term memory,or even electronic documentation.  Maybe you threw a few comments into your code but those, too, are probably obsolete.  Even your own scribbled notes you cribbed while developing the product look like someone else’s writing.  In another language.

Sales and Marketing folks come back from a leisurely client lunch, belching cocktails and talking about their next golf outing while you’re digging melted cheese from your microwaved hot-pocket out from between the keys of your keyboard.  The next time you look up from your coding jag, it’s because the janitor has startled your while reaching for your full (of crap-food wrappers and amazon packing) trashcan and you realize that, once again, you’re headed for another all-nighter.

So, the next time I go on a job interview, and some smarmy, fresh-out-of-college twenty-something manager asks me about “design patterns”, don’t be surprised when I get up and walk out on you.

See, I’m a working software engineer that lives in the real-world.  My only concern is the monetization of my product.  I know that the only way I’m going to retire early is if my product becomes the NEXT BIG THING.  I work at a start-up — my stock is pretty much going to be worthless because it’s been so heavily watered down through seed investor funding.  And I have no expectations of a stock based pay-day.  Not in this economy.  You’re good at match — you figure out the odds of being the next Facebook, Google, or Zynga.

My only chance to take that beach-front retirement package is through the executive exit-plan for the company.  See, I don’t want to become one of the nameless horde working at Zynga, Facebook or Google.  I’m here to contribute heavily to the success of the product; my opinions and my work matters at every level of my involvement and if I were to suddenly leave the company, turmoil would follow in my wake.  I am not an afterthought represented by the CC: tag line in a product email.  I am in the To: block always.

If you think design patterns, O(N) search algorithms are important enough to ask as  interview questions, then I don’t want to work for you because I don’t feel that you have a grasp of what’s important.

The bottom line is the bottom line.  It’s as simple as that.  If you’re not focused on the monetization, the promotion, the saturation, of my product, in the ethersphere,  then you’re only in my way.

I’m a software engineer in the real world.

Part 3: Creating Linux Development Environment (PHPStorm and the JDK)

Category : Technical
No Gravatar

I stopped working yesterday on the installation because I hit a pothole installing PHPStorm by JetBrains.

As I mentioned in the previous article, and in case you’re just tuning in, I am first working towards a LAMP development environment on an older PC running 64-bit Linux.  We’ve decided on CentOS 6 as the base distribution and installed the LAMP stack yesterday.  I installed the PHPStorm package but hit a snag when I received an error message telling me that it required the JDK runtime … thingys.  (Whatever – I assiduously avoid Java.)

I installed the openjdk packages with yum and got PHPStorm to start-up, albeit with many dire warnings and threats to the graphics system.  Apparently PHPStorm is comfortable running only with the jdk from SUN/Oracle.

I then downloaded and RPMd the SUN/Oracle version of the jdk and restarted.  What happened next were error messages telling me that I need to set-up the java (dk) environment correctly as, now, the two were conflicting with each other.

ERROR: Cannot start WebIde.
No JDK found to run WebIde.  Please validate either WEBIDE_JDK, JDK_HOME or JAVA_HOME environment variable points to valid JDK installation.

See, in the linux world, the PHPStorm is launched from a shell script.  It checks your environment for the JDK through these variables and, if correctly defined, launches the IDE.

There’s a java-sdk configuration file located as /etc/java/java.conf – don’t make the same mistake I made and edit this file to re-direct/create environment  variables so they point to the SUN/Oracle version of the JDK.

The SUN/Oracle version of the Java SDK installed in: /usr/java/jdk1.7.0/ which will change for your system depending, I’d assume, on your distribution and version of the SDK.

To reconcile the conflicts, I used the yum installer to remove any traces of the openjdk — all packages were removed and then I did a yum clean all to reset the environment.

Since I’m the only user on this system, I next cd’d into my home directory and pulled up the .bashrc file – this will modify the bash shell environment for every terminal session I start.  I added the following two lines to the .bashrc:

1
2
JDK_HOME=/usr/java/jdk1.7.0
export JDK_HOME

I exited the editor and reloaded my bash environment:

# . ./.bashrc

From there, all I need to do is start the PHPStorm shell script which launches the application and I’m good to go!

You can install the PHPStorm folder anywhere.  Using your bashrc file, you can make an alias to the start-up shell script so that you can launch the IDE anywhere from the CLI environment.

alias phpstorm='nohup /home/user/folder/PhpStorm/bin/PhpStorm.sh &amp;'

The nohup allows the program to ignore SIGHUP — in other words, if you close the terminal from where you launched PHPStorm, you will not close PHPStorm as well.  The ampersand (&) at the end of the command tells the shell interpreter to launch the application as a “background” task which frees up your terminal session so that you can continue to use the shell while PHPStorm is running.

At this stage, I’m pretty much good to go for basic LAMP development.  I’ve got a running mySQL server, Apache2 is good to go, and PHP5 is installed.  I will enhance my environment by adding a few packages such as:

I also want to give some thought to virtual hosts — I’ll cover this topic in a future post — within my local Apache2 environment, I’m going to want to establish several different virtual host environments, each of which point to a different documentRoot location (or code repository) depending on which application/environment I’m currently working on.
I’ll also have to plan my filesystem repositories carefully — for the most part, I’ll be working as a subversion server for home-projects, while also working as a subversion client for work projects.
Which reminds me – in the first article in this series, I reported on the filesystem utilization following a clean Fedora 15 install.  Here’s the state of the current filesystem (CentOS 6) following the LAMP stack install, and the install of the PHPStorm IDE and the SUN/Oracle JDK:
/ (root):   50gb, used 6%, 47gb available
/boot:      485mb, used 11%, 409mb available
/home:    864gb, used 1%, 820gb available
I’m looking pretty good for user filesystems but I’ll want to check my mySQL configuration and ensure that databases are being created in the /home filesystem and not in the /root filesystem also.
Ok – done for this weekend and off to play some Rift!  Hope this helps someone!
PS: If you want some detailed tutorials on installing any of the supplemental packages I listed at the end of this article, please leave a comment!

Page optimized by WP Minify WordPress Plugin

Weather forecast by WP Wunderground & Denver Snow Plowing