LAMP | shallop.com

Testing Arrays in PHP – Back to Basics…

Posted by : Micheal Shallop Posted on : 20-12-2012

(No Ratings Yet)

Loading ...

Category : Technical

Tags: arrays, count, decisions, empty, functions, isset, is_null, LAMP, PHP, programming, results, testing

Sometimes, when you’re wallowing through your abstraction class layers, you find yourself using code for simple functions that are normally the focus of an Intro to Programming class. The problem is that you spend so much time on advanced topics, that your common-knowledge of simpler functions becomes… murky… You have to stop what your’re doing and review the manual for simple functions just to convince yourself that the results you expect are the results you’re getting in your code.

Sound familiar?

I recently encountered just this issue while evaluation whether or not my data structures were populated by testing the structure against several PHP built-in functions designed to inform me of the status of the structure. The data structure in-question was just a multi-dimensional array or arrays used to store non-relational records within a mongo collection. Because the data itself is an array, and each element can also be an array, the code to support the structure is all recursive when it comes to structure traversal and validation.

So the testing of the elements within the array boiled down to a simple detection to determine if I was dealing with a data element (end-point) or another array of arrays.

PHP offers several built-in functions for this type of testing:

isset() – determine if a variable is set and not null
is_null() – determine if a variable is null
empty() – determine if a variable is empty: does not exist or if it’s value is FALSE

What I found myself doing was writing conditionals that used combinations of these three functions. Reviewing my code later, when I asked myself if this was really necessary, I was somewhat embarrassed by the realization that I couldn’t conclusively say. I mean, I know what these functions do, right? I use them daily… but I didn’t have 100%-confidence that I was using them correctly within the context that I needed.

Reviewing the documentation on the functions (I provided the PHP.net links above), I decided that I should study the problem further, write some test stubs, and see what PHP would tell me with the return values from these methods. I thought the results were interesting enough to share, hence this article.

Using JSON notation, let’s create a simple data structure:

foo = ( {
 { 'fname' : 'fred', 'lname' : 'flintstone', 'city' : 'bedrock' },
{ 'fname' : 'fred', 'lname' : 'flintstone', 'city' : 'bedrock' }
} )

In PHP, this declaration would appear like so:

$bar = array('fname' => 'fred', 'lname' => 'flintstone', 'city' => 'bedrock');
$foo = array($bar, $bar);

If we var_dump() the contents of the array, along with their associated indexes, we get this output:

Array
(
    [0] => Array
        (
            [fname] => fred
            [lname] => flintstone
            [city] => bedrock
        )
    [1] => Array
        (
            [fname] => fred
            [lname] => flintstone
            [city] => bedrock
        )
)
Complete Array (foo) with (bar) count: 2
array[0] count: 3
array[1] count: 3

So far, pretty simple. Let’s pose these questions to see how the PHP built-in functions would handle the following scenarios:

If I unset() the fname in $foo[1], will the count of the $foo[1] change?
If I unset() all of the $foo[1]‘s indexes (lname, city), what will the count of $foo[1] be?
What does $foo look like now?
- Does $foo[1] test true for isset()?
- Does $foo[1] test true for is_null()?
- Does $foo[1] test true for empty()?
What does $foo look like if I unset $foo[1]?
- Does $foo[1] test true for isset()?
- Does $foo[1] test true for is_null()?
- Does $foo[1] test true for empty()?
What is the count() of $foo?
If I set $foo[1] to null, do my results change?
- Does $foo[1] test true for isset()?
- Does $foo[1] test true for is_null()?
- Does $foo[1] test true for empty()?
Does setting $foo[1] to null count()?

I wrote a small test-stub to answer all of these questions (listing is at the end of this article) and the results I got back were interesting. Let’s take a look by answering all of the questions posed above:

The array count of $bar, as expected, is: 2. Additionally, each sub-array within $bar has a count of: 3. No surprises.

After I unset($foo[1]['fname']), the count of $foo[1] drops to: 2. Not surprised.

After I unset the remaining indexed values in $foo[1] (lname, city), the count of $foo[1] is now: 0. Not surprised.

The count($foo) however is still: 2. Even though all of the indexed elements in $foo[1] were unset, the placeholder for $foo[1] still exists and counts. This told me that, in a scenario where you may be stripping invalid elements from an input array, in the border-case where all of the elements are removed, using count() on the parent (containing) array will not indicate this. Interesting.

$foo[1] tests TRUE for isset() — not surprising given the above.

$foo[1] tests TRUE for is_null() — not surprising considering the container index is basically pointing to nothing (zero elements).

$foo[1] tests TRUE for empty() — as expected.

Next, I remove the $foo[1] from the $foo array using the unset() function. The var_dump() of $foo now shows only our first element which is still untouched as the only member within the array. The count() of $foo is now: 1.

We apply the same series of tests and get:

$foo[1] tests FALSE for isset() and, bonus, generates a PHP notice because the offset index no longer exists. Interesting.

$foo[1] tests TRUE for is_null(). Not surprised.

$foo[1] tests TRUE for empty(). Again, not surprised.

Finally, we’re going to set $foo[1] to NULL under the assumption that we don’t want to lose the index position of the data set during processing providing a means for quickly detecting which elements in the input data were completely discarded.

$foo[1] is now set to NULL with the following results:

$foo[1] tests FALSE for isset(). Sort of interesting in that we have a placeholder which can be counted, but since it is NULL value, it’s not “set” or present.

$foo[1] tests TRUE for is_null(). Expected.

$foo[1] tests TRUE for empty(). As documented.

Interesting, the count($foo) is back to: 2, even though the $foo[1] index is null, and no data, and tests FALSE for isset().

These results caused the generation of the following matrix:

Checked-boxes indicate a YES or TRUE answer while unchecked-boxes are NO or FALSE.

Of the functions tested, empty() appears to provide the best across-the-board response to decision testing with code for determining the presence (or absence) of array content as a stand-alone test.

Which leaves the question that started this entire trip down the rabbit-hole: do I need to use multiple functions to test the state of an array or sub-array?

Probably not, in most cases – I can shave a few ticks here and there from my code by reducing the complexity of my decision statements by eliminating redundant functions as long as I keep tight controls over my expectations of the data itself, the structure being used, and the state of the indexes.

This code was tested on an Ubuntu 12.10 install using PHP 5.4.6 (cli). I hope you found this experiment as interesting as I did; it’s good to go back and review the basics so you KNOW what’s going in in your code…

Happy Coding!

Program Listing:

<?php
$bar = array('fname' => 'fred', 'lname' => 'flintstone', 'city' => 'bedrock');
$foo = array($bar, $bar);
print_r($foo);
echo 'Complete Array (foo) with (bar) count: ' . count($foo) . PHP_EOL;
for ($index = 0; $index < count($foo); $index++) {
    echo 'array[' . $index . '] count: ' . count($foo[$index]) . PHP_EOL;
}
echo 'Nuking index 1 of foo...' . PHP_EOL;
unset($foo[1]['fname']);
echo 'unset fname...count of array[1] is: ' . count($foo[1]) . PHP_EOL;
unset($foo[1]['lname']);
unset($foo[1]['city']);
echo 'all members unset... count: ' . count($foo[1]) . PHP_EOL;
print_r($foo);
if (isset($foo[1])) {
    echo 'foo[1] tests true for isset()' . PHP_EOL;
} else {
    echo 'foo[1] tests false for isset()' . PHP_EOL;
}
if (is_null($foo[1])) {
    echo 'foo[1] tests true as null' . PHP_EOL;
} else {
    echo 'foo[1] tests false as null' . PHP_EOL;
}
if (empty($foo[1])) {
    echo 'foo[1] tests true for empty' . PHP_EOL;
} else {
    echo 'foo[1] tests false for empty' . PHP_EOL;
}
echo 'unset foo[1]...' . PHP_EOL;
unset($foo[1]);
print_r($foo);
if (isset($foo[1])) {
    echo 'foo[1] tests true for isset()' . PHP_EOL;
} else {
    echo 'foo[1] tests false for isset()' . PHP_EOL;
}
echo 'is_null($foo[1]) generates a PHP Notice:' . PHP_EOL;
if (is_null($foo[1])) {
    echo 'foo[1] tests true as null' . PHP_EOL;
} else {
    echo 'foo[1] tests false as null' . PHP_EOL;
}
if (empty($foo[1])) {
    echo 'foo[1] tests true for empty' . PHP_EOL;
} else {
    echo 'foo[1] tests false for empty' . PHP_EOL;
}
echo 'count of foo: ' . count($foo). PHP_EOL;
echo 'setting foo[1] to null...' . PHP_EOL;
$foo[1] = null;
print_r($foo);
if (isset($foo[1])) {
    echo 'foo[1] tests true for isset()' . PHP_EOL;
} else {
    echo 'foo[1] tests false for isset()' . PHP_EOL;
}
if (is_null($foo[1])) {
    echo 'foo[1] tests true as null' . PHP_EOL;
} else {
    echo 'foo[1] tests false as null' . PHP_EOL;
}
if (empty($foo[1])) {
    echo 'foo[1] tests true for empty' . PHP_EOL;
} else {
    echo 'foo[1] tests false for empty' . PHP_EOL;
}
echo 'does null count?' . PHP_EOL;
echo 'count of foo: ' . count($foo). PHP_EOL;

Program Output:

Array
(
    [0] => Array
        (
            [fname] => fred
            [lname] => flintstone
            [city] => bedrock
        )
    [1] => Array
        (
            [fname] => fred
            [lname] => flintstone
            [city] => bedrock
        )
)
Complete Array (foo) with (bar) count: 2
array[0] count: 3
array[1] count: 3
Nuking index 1 of foo...
unset fname...count of array[1] is: 2
all members unset... count: 0
Array
(
    [0] => Array
        (
            [fname] => fred
            [lname] => flintstone
            [city] => bedrock
        )
    [1] => Array
        (
        )
)
foo[1] tests true for isset()
foo[1] tests false as null
foo[1] tests true for empty
unset foo[1]...
Array
(
    [0] => Array
        (
            [fname] => fred
            [lname] => flintstone
            [city] => bedrock
        )
)
foo[1] tests false for isset()
is_null($foo[1]) generates a PHP Notice:
PHP Notice:  Undefined offset: 1 in /home/Namaste/utilities/unsetTest.php on line 41
foo[1] tests true as null
foo[1] tests true for empty
count of foo: 1
setting foo[1] to null...
Array
(
    [0] => Array
        (
            [fname] => fred
            [lname] => flintstone
            [city] => bedrock
        )
    [1] =>
)
foo[1] tests false for isset()
foo[1] tests true as null
foo[1] tests true for empty
does null count?
count of foo: 2

Don't miss a single post! Subscribe to my RSS feed

Press My Buttons !

Posted by : Micheal Shallop | Leave a Comment |

PHP: Comparing Object Structures

Posted by : Micheal Shallop Posted on : 21-08-2012

(No Ratings Yet)

Loading ...

Category : Technical

Tags: clones, comparisons, copies, development, equals, identities, LAMP, object, objects, oop, PHP, program, programming, triple-equals

I’m working on a project where I am converting an established REST API over to a rabbitMQ service. Because, you know, dinosaur, I’m continuing to use PHP as my language of choice for both the rabbitMQ producers and consumers. (Sorry – just not there with Python. Yet.)

Anyway, as part of the architecture, I’m building a rabbitMQ service to handle all incoming requests that would normally be handled as POST requests thought the REST API.

Our data flow necessitates the rapid store of several different types of objects loosely associated with real-world events and objects: users, registration, browser events, etc. You get the idea: the data is different but kind-of the same…different enough to require different class objects (because of the data retrieval or storage requirements) yet close enough that, as the programmer, you’re constantly plagued by deja-vu when it comes to coding the class methods.

The development team collectively decided to make the move to a queue-messaging system so I evaluated and chose RabbitMQ as the framework to use to convert my existing REST API.

Without going too deep into details (a future article and I want to keep this post somewhat easy), I have the option of pooling the collection of incoming data that, under REST (segregated based on object functionality: members, subscriptions, payments, etc.), into a single collection point. In other words, using REST, each real-world object has a dedicated service which is represented in the REST/data stack as a REST service attached to a PHP class object which is extended from a base data class dependent on mongo which, in turn, is extended from a base data handling class (auditing, memcache, etc.).

What that means is that if I have 20 or so services offered via REST, I probably have 20 different REST map files that process a POST request that can instantiate up to 20 different object classes unique to the service being offered. No problem – but under RabbitMQ, this can be done better.

The problem/challenge I encountered (I’m intentionally skipping ahead here because I don’t want this article to be about either RabbitMQ or REST) is that I wanted establish a single consumer service for accepting and storing the input data for all of my object classes. In other words, for any incoming data to be stored, I want to be able to peek at the structure being used to store the data and then invoke the proper class to handle data validation and storage.

So … the point is, I want my RabbitMQ queue consumer to evaluate the incoming data structures and determine which data classes I’ll need to work with and to dispatch the work accordingly to the class handlers.

Comparing objects is the point of this article — if I get an object for a user, I want to invoke the user class. If I get a registration data-packet, the registration class and so-on…

This, in-turn, led me down the rathole of PHP object comparison and I picked-up a few things to share with you…so, at least, we get to the point of the article.

When comparing PHP objects, you usually have one of two goals in-mind: I want to see if the object contents are the same, or I want to see if the object structures are the same.

For the purposes of my program, I want to evaluate the structure itself and dispatch the data to the appropriate class handler; I don’t care about the contents — that’s the designated responsibility of said class handler.

But, fear not, I will talk about comparing PHP objects as it relates to the contents. For this, we’ll use the ‘==’ and ‘===’ operators.

Conceptually, I am going to create two base classes, named cat and dog. From these classes we’ll instantiate variable structures and compare these structures against each other. The code to do all this is at the bottom of this article so here’s the walk through:

The Plan

I am going to instantiate two variables: $siamese and $persian of members of the cat class.

I am going to instantiate one variable: $husky as a member of the dog class.

I am going to clone the variable $malamute from $husky.

Both dog and cat have identical class variables and methods. (For simplicity, one of each.)

After I create my variables, I am going to compare them (within their classes) for equality (‘==’) and then compare them for identity (‘===’).

Next, I will increment $siamese and $husky and repeat the above checks.

Next, I will declare a copy of $husky and name him $einstein (my dog!) and repeat the checks.

Then I will determine who’s a cat and who’s a dog — looking at the structure of the class and not the contents.

And, finally, I will var_dump the relative structures so you can see the contents.

The Results

In the initial comparisons of $siamese to $persian, the comparison operators evaluate true but the identity operators evaluate false. This makes logical sense: Siamese and Persians are both cats, but this Siamese has a different identify from this Persian.

When I increment the member variable ($a) within $siamese and re-run the comparison, both checks now evaluate false. The Siamese no longer has the same contents as the Persian and they still don’t have the same identity.

For sanity, I check to ensure that a $husky is not a $persian in both comparison and identity checks and, thankfully, it’s not.

Next, I clone $malamute from $husky and run the basic checks:

The Malamute has the same contents as the Husky, but they do not have the same identity.

Next, let’s create $einstein by making him a copy of $husky with the assignment (‘=’) operator and run the comparisons — no surprise: Einstein has the same contents as Husky and has the same identity. The identity check evaluated true because both objects share the same memory space and this is shown by:

I call the method in $einstein to increment the member variable by one and re-run the comparison: both checks again evaluate to true because we’re manipulating shared memory space objects. In other words, although both $einstein and $husky have the same member variable ($a), both variables are pointing to the same physical storage location in memory. So, if I change the contents of $a in $einstein, the contents of $a in $husky changes as well.

By var_dump’ing (dispaying the contents) of these variables, I can see that the $husky’s member variable ($a) has been incremented even though the increment operation was done by the $einstein object.

Finally, let’s use the PHP function instanceof to evaluate the type of class we have. This is the dispatch mechanism I am using in RabbitMQ to invoke the correct object class for processing my incoming data.

I use instanceof to ask if $siamese is a dog or $husky is a cat. Pretty straightforward — and I get the expected results in the output.

Wrap-Up

So, hopefully, you know when to use ‘==’ and ‘===’ when comparing PHP objects and what an object identity is, and the difference between comparing the contents of a class versus evaluating the structure of a class.

Source code and sample output below…hope this helps!

Source code

<?php
 class cat
 {
 public $a = 1;
 public function addone()
 {
 $this->a = ++$this->a;
 }
 }
 class dog
 {
 public $a = 1;
 public function addone()
 {
 $this->a = ++$this->a;
 }
 }
 $siamese = new cat();
 $persian = new cat();
 $husky = new dog();
 $malamute = clone $husky;
 $einstein = $husky;
 if ($siamese == $persian) {
 echo 'siamese == persian' . PHP_EOL;
 }else {
 echo 'siamese != persian' . PHP_EOL;
 }
 if ($siamese === $persian) {
 echo 'siamese === persian' . PHP_EOL;
 } else {
 echo 'siamese !== persian' . PHP_EOL;
 }
 echo 'calling siamese->addone()' . PHP_EOL;
 $siamese->addone();
 if ($siamese == $persian) {
 echo 'siamese == persian' . PHP_EOL;
 } else {
 echo 'siamese != persian' . PHP_EOL;
 }
 if ($siamese === $persian) {
 echo 'siamese === persian' . PHP_EOL;
 } else {
 echo 'siamese !== persian' . PHP_EOL;
 }
 if ($husky == $persian) {
 echo 'husky == persian' . PHP_EOL;
 } else {
 echo 'husky != persian' . PHP_EOL;
 }
 if ($husky === $persian) {
 echo 'husky === persian' . PHP_EOL;
 } else {
 echo 'husky !== persian' . PHP_EOL;
 }
 echo 'clone wars...' . PHP_EOL;
 if ($husky == $malamute) {
 echo 'husky == malamute' . PHP_EOL;
 } else {
 echo 'husky != malamute' . PHP_EOL;
 }
 if ($husky === $malamute) {
 echo 'husky === malamute' . PHP_EOL;
 } else {
 echo 'husky !== malamute' . PHP_EOL;
 }
 echo 'playing with Einstein: ' . PHP_EOL;
 if ($einstein == $husky) {
 echo 'einstein == husky' . PHP_EOL;
 } else {
 echo 'einstein != husky' . PHP_EOL;
 }
 if ($einstein === $husky) {
 echo 'einstein === husky' . PHP_EOL;
 } else {
 echo 'einstein !== husky' . PHP_EOL;
 }
 echo 'calling einstein->addone()' . PHP_EOL;
 $einstein->addone();
 if ($einstein == $husky) {
 echo 'einstein == husky' . PHP_EOL;
 } else {
 echo 'einstein != husky' . PHP_EOL;
 }
 if ($einstein === $husky) {
 echo 'einstein === husky' . PHP_EOL;
 } else {
 echo 'einstein !== husky' . PHP_EOL;
 }
 if ($siamese instanceof cat) echo 'siamese is a cat' . PHP_EOL;
 if ($siamese instanceof dog) echo 'siamese is a dog' . PHP_EOL;
 if ($husky instanceof cat) echo 'husky is a cat' . PHP_EOL;
 if ($husky instanceof dog) echo 'husky is a dog' . PHP_EOL;
 echo 'var-dumps:' . PHP_EOL . 'husky: ' . PHP_EOL;
 var_dump($husky);
 echo PHP_EOL . 'malamute: ' . PHP_EOL;
 var_dump($malamute);
 echo PHP_EOL;

Output:

Source code

siamese == persian
 siamese !== persian
 calling siamese->addone()
 siamese != persian
 siamese !== persian
 husky != persian
 husky !== persian
 clone wars...
 husky == malamute
 husky !== malamute
 playing with Einstein:
 einstein == husky
 einstein === husky
 calling einstein->addone()
 einstein == husky
 einstein === husky
 siamese is a cat
 husky is a dog
 var-dumps:
 husky:
 object(dog)#3 (1) {
 ["a"]=>
 int(2)
 }
 malamute:
 object(dog)#4 (1) {
 ["a"]=>
 int(1)
 }

Don't miss a single post! Subscribe to my RSS feed

Press My Buttons !

Posted by : Micheal Shallop | Leave a Comment |

Why is my mongo query so slow?

Posted by : Micheal Shallop Posted on : 19-12-2011

(No Ratings Yet)

Loading ...

Category : Technical

Tags: architecture, btree, collection, cursor, database, dba, development, documents, explain, indexing, LAMP, millis, mongo, mongodb, nscanned, optimization, plan, programming, query, search, technical, tutorial

Why’s my mongodb query so slow?

I got my geospatial collection set-up — I am running some really great queries making sure that the locations I am pulling aren’t in any sort of cache, and I am just blown-away by how fast data is being returned.

The problem is: when I query the collection to pull up the requisite lon/lat data by name: city & state, or city & country, the query seems to take seconds to complete!

I set-up the table correctly…I indexed the crap out of all my columns…a week or two ago, I was at the mongoSV 2011 in Santa Clara and learned some really cool stuff about queries, indexing, and performance management, so let’s dig-out the notes and see where I went wrong. Because I strongly doubt that the problem is in mongo but, rather as we used to say in technical support: this is a PBCK issue…

The first thing I want to do is run an explain against my query so I can see mongo’s query plan for my query. This should provide me with a starting point for trying to figure out what went wrong.

> db.geodata_geo.find({ cityName : “Anniston”, stateName : “Alabama” }).explain();

By adding the trailing function: .explain(), I’m requesting that mongoDB return the query-plan to me instead of executing the query. I hit enter to launch the explain() and get back the following output:

> db.geodata_geo.find({ cityName : “Anniston”, stateName : “Alabama”}).explain();
{
“cursor” : “BasicCursor“,
“nscanned” : 3691723,
“nscannedObjects” : 3691723,
“n” : 1,
“millis” : 2269,
“nYields” : 0,
“nChunkSkips” : 0,
“isMultiKey” : false,
“indexOnly” : false,
“indexBounds” : {
}
}

The important information, I bold-faced in the query output (above). What this output is telling me is that I’ve using a “BasicCursor” for my search cursor — which is indicates that, yes, I am doing a table-scan on the collection. So, already I know my query is not optimal. But, wait! More good news…

The value for nscanned and nscannedObjects is the same: 3,691,723 — which coincidently is the same as the cardinality of the collection. This number is the number of documents scanned to satisfy the query which, given it’s value, confirms that I am doing a full table scan.

millis tells me the number of milliseconds that the query would take: 2.269 seconds: way too slow for my back-end methods() serving a REST API — unacceptable.

And then we get to the tell: IndexOnly tells me that if the query could have been resolved by an (existing) covering index. Seeing the value false here tells me that the collection has no index on the columns I am scanning against.

What?!? I know I indexed this collection…

So, I run db.geodata_geo.getIndexes() to dump my indexes and … I … don’t see my name columns indexed. Oh, I remembered to index the the ID and Code columns…but somehow, indexing the Name columns completely slipped past my lower brain-pan.

I add these indexes to my collection:

> db.geodata_geo.ensureIndex({ cityName : 1 });
> db.geodata_geo.ensureIndex({ stateName : 1 });

And then I rerun the query plan and see the following output:

> db.geodata_geo.find({ cityName : “Anniston”, stateName : “Alabama”}).explain();
{
“cursor” : “BtreeCursor cityName_1″,
“nscanned” : 2,
“nscannedObjects” : 2,
“n” : 1,
“millis” : 101,
“nYields” : 0,
“nChunkSkips” : 0,
“isMultiKey” : false,
“indexOnly” : false,
“indexBounds” : {
“cityName” : [
[
"Anniston",
"Anniston"
]
]
}
}

Instead of BasicCursor, I see BtreeCursor which gives me a happy. I also see that the nscanned and nscannedObjects values are now more realistic…seriously: 2 is a LOT better than 3.6 million something, right? Another happy for me!

I score the third happy when I see that the millis has dropped down to 101: 0.101 seconds to execute this search/query! Not jaw-dropping, I agree — but acceptable considering that everything is running off my laptop…I know production times will be much, much lower.

In the end, I learned that a simple tool like .explain() can tell me where my attention is needed when it comes to optimization and fixing even simple, seemingly innocent queries. Knowing what you’re looking at, and what you’re looking for, is pretty much thick-end of the baseball bat when it comes to crushing one out of the park.

I hope this helps!

Reference Link: Explain

Don't miss a single post! Subscribe to my RSS feed

Press My Buttons !

Posted by : Micheal Shallop | Leave a Comment |

Searching MongoDB Sub-Documents…

Posted by : Micheal Shallop Posted on : 15-12-2011

(No Ratings Yet)

Loading ...

Category : Technical

Tags: collection, database, document, how-to, LAMP, mongo, mysql, programming, search, sub-document, technical, tutorial

I’ve recently finished a mongo collection that stores all auditing data from my application — specifically, it records every database transaction, conducted in either mySQL or mongo, assigning an event-identifier to the event, and storing the data under an event ID within a single sessionManger object.

Sounds good?

Well, I like it. This design eliminated the need to maintain meta-data in my data tables since I can pull transaction history for any record that I’ve accessed.

The problem is that, being new to mongodb, accessing what I’ve put into mongodb isn’t (yet) as intuitive as, say, my mySQL skills are.

Sub-documents within a mongo document are analogous to the results of a mySQL join. One of the key motivators in storing this information in mongodb to begin with was that I could de-normalize the data by storing the sub-document with it’s parent instead of having to incur the expense of a search-join-fetch later.

Traditionally, any data objects defined as a one-to-many type of a relationship (1:m) were stored in multiple mySQL tables and were accessed via some sort of join mechanism.

Mongodb breaks that traditional mold by allowing you to store a sub-document (the “m” part of the 1:m relationship) within the same document in which you’re currently working.

Using my sessionManger document, I have a document that looks something like this:

{
_id : somevalue,
foo : bar,
event : {},
argle : bargle,
}

My desire is to, for every database event that is recorded, enter information about that event within the sub-document that I’ve wittily named “event”.

In my PHP code, I’ve written a sequence manager for mongo that maintains a document containing sequence values for various tables. Think of this as the functional version of mySQL’s auto-increment feature. I decided, then, for the sessionManager events, I would use this key sequence to obtain unique values and use those as my sub-document index. I’d then store whatever data I needed to store using the sequence value as a sub-document key, or index:

{
_id : somevalue,
foo: bar,
event : {
n : {
created : dateval,
table : tableName,
schema : dbSchema,
query : lastQuery
}
}
argle : bargle
}

So, when I need to add another event, I just create a new sub-document under the event key, then add the data I need to store under the sub-document index key.

Worked like a champ!

And then I asked myself: ”So, Brainiac, how would you go about extracting event -n- from your collection?”

I went through a lot of failed query attempts, bugged a lot of people, googled and saw stuff that led me down many plush ratholes until I finally, through some serious trial-and-error, got the answer…

> db.mytable.find( { foo : bar }, { ‘event.n’ : 1 } );

where n = the number of the event I want to find.

If I want to get all of the events for a particular document (sessionManger object), then I would write something like:

> db.mytable.find( {foo : bar}, { event : 1});

If I wanted to return all of the events for all of the objects, then I would write this:

> db.mytable.find( {}, {event : 1});

What I’ve not been able to figure out, so far, is how I can use $slice to grab a range of events within a document. Everything I try returns the full sub-set of documents back to me. The doc tells me that $slice is used to return a subrange of array elements, which is what I thought “event.n” was but, apparently, it’s not. (I think it’s an object (sub-document) which is why $slice fails for me.)

It’s not a big deal because, programmatically, I can grap the entire sub-document from it’s parent and parse in-memory to get the desired record. And, if I know what the value for -n- is, then I can fetch just that one sub-document. So, I’m ok for now. However, please feel free to enlighten me with your expertise and experience should you see where I am failing here, ok?

Don't miss a single post! Subscribe to my RSS feed

Press My Buttons !

Posted by : Micheal Shallop | Leave a Comment |

mongodb.findOne() — calling with PHP variables (not literals)

Posted by : Micheal Shallop Posted on : 21-10-2011

(No Ratings Yet)

Loading ...

Category : Technical

Tags: array, database, findone, LAMP, mongo, mongodb, mysql, object, oop, oriented, PHP, programming, query

So I’ve been doing a lot of work, for work, in MongoDB lately and I’ve learned an awful lot. Or, depending on your point of view, a lot that’s just awful.

See, there’s not what you could even charitably call a lot of MongoDB documentation to begin with. If you filter what is available on, oh, say, PHP implementation, well the results just dwindle to something roughly the same size as a tax-collector’s heart.

Here’s the scenario — I’ve been working on adding a mongo abstraction class on top of my base-data abstraction class — whereas said classes are extended by the table-level class instantiation. This allows me to keep all of my query logic in the middle tier of the class design, generic and administrative functions in the base class, and table-specific stuff in the table class. So far, so good, right?

Well, I get the mongo constructor running and, like it’s mySQL counterpart, I have an rule in every table constructor that states “if I pass a indexed field and it’s value to the constructor, then instantiate the class pre-populated with that record.”

And that’s where things start to head south.

In my constructor logic, I’m only allowing single-value key->value pairs as constructor parameters with the design intention of getting a record from the db using the pkey of the table/collection. In other words, you get one column and one column value. So, if you’re going to instantiate a new user object, you’d probably want to pass-in the primary-key field of a user and that field’s value:

$objUser = new UserProfile(‘email’, ‘mshallop@gmail.com’); // instantiate a new user object with this email address

Still pretty easy. I bang out the mySQL equivalent in nothing flat. I hit a huge pothole when I get to the mongo side.

The method is defined as a protected abstract method in the base class – so this method has to appear in both child classes as defined in the parent:

protected abstract function loadClassById($_key, $_value);

So I have my methods defined in both the mySQL and mongoDB middle layer. My strategy for the mongo fetch-and-return is pretty simple — once the class has been instantiated, do the following:

make sure the $_key value exists in the allowed field list
make sure the $_value has a value
query mongodb using .findOne()
store the return key->value pairs in the member array
return status

That’s pretty much it. But I run into huge problems when I get to step 3 — use the mongoDB findOne command.

The findOne method takes an array input of the key->value pair. From the mongo command line, you’d execute something like this:

> db.session_ses.findOne({'idpro_ses' : 1})
{
 "_id" : ObjectId("4ea1af93ddc69802376b56d1"),
 "id_ses" : 1,
 "idpro_ses" : 1
}

( Just to show you that the data exists in the mongo collection…)

But, the PHP-ized version of the method is a wee bit different:

$this->collection->findOne(array(‘idpro_ses’ => 1));

All of the examples that I’ve been able to locate show using the method by invoking it using literals. My problem is that I have the two input parameters sent to the method ($_key and $_value) and I’ve got to find a way to get the PHP version of the method call to work using variables instead of constants. This is what didn’t work:

~~$this->collection->findOne(array($_key => $_value));~~

~~$this->collection->findOne(array(“‘” . $_key . “‘” => $_value));~~

~~$this->collection->findOne(array(“{$_key}” => $_value));~~

~~$aryData = array();~~
~~$aryData[$_key] = $_value;~~
~~$this->collection->findOne($aryData);~~
or
~~$this->collection->findOne(var_dump($aryData));~~

I thought this worked but I was wrong:

$this->collection->findOne(array(array_keys($aryData) => array_values($aryData)));

This format returned a mongo record — the problem was that it returned the first mongo record independently of any key-search criteria.

What finally worked for me was this:

            $qs = array(); // QueryStructure
            switch($this->fieldTypes[$_k]) {
                case 'int' :
                    $_v = intval($_v);
                    break;
                case 'str' :
                    $_v = strval($_v);
                    break;
                case 'float' :
                    $_v = floatval($_v);
                    break;
            }
            $qs[$_k] = $_v;
            $aryData = $this->collection->findOne($qs);

[Update]

I encountered a similar problem when trying to update records in a mongo collection — while I could update the record from the mongo command line, I did not experience the same success in trying to execute the command from within my PHP program…

$foo = $collection->find(array(‘id_geo’ => $row['id_geo']));

Consistently failed. No exceptions were caught, and mongo’s findLastError() reported no errors in the transaction.

After several iterations of debugging and attempting various work-arounds, I stumbled upon the solution as being one of casting. While the variable was being evaluated in the PHP array as type int, somehow this wasn’t being interpreted that way by Mongo. Casting the variable to an integer:

$foo = $collection->find(array(‘id_geo’ => intval($row['id_geo'])));

generated a successful query for both the find() and my update() functions.

As I gain experience with Mongo, I expect to discover more of these little mannerisms…

Don't miss a single post! Subscribe to my RSS feed

Press My Buttons !

Posted by : Micheal Shallop | Leave a Comment |

Web Services with PHP & nuSoap – Part 1.1

Posted by : Micheal Shallop Posted on : 26-09-2011

(1 votes, average: 5.00 out of 5)

Loading ...

Category : Technical

Tags: apache, client, code, curl, http, LAMP, newsoap, nusoap, PHP, programming, rpc, server, services, soap, source, technical, tns, tutorial, web, webservices, wsdl, XML, xsd

Introduction

[EDIT] – This is a re-hash of a document I wrote a couple years ago. There’s been changes to the nuSOAP library and I wanted to document the updates relative to the tutorial series. Also, I need to fix a lot of the broken-links and code listings since I’ve changes hosting providers since this article was first written.

This article is specific to nuSOAP release 0.9.5 on 2011-01-13. This tutorial was updated on September 22, 2011.

Once upon a time, I was tasked with developing a web-services by my boss as the integration point between our production application and Sales Force. At this point, although I’d heard of web services … kind of … didn’t Amazon use web services for something? Still, I’d never coded a web-services based application before.

At this point, I have to assume you are unfamiliar with the concept of web services and why you may have to create and provide a web-services offering to your client base. Web services allow a remote client to access functionality (as defined by you, the programmer) via the standard HTTP interface (normally, port:80) to your application and database services.

Back in the day, networking services (semaphores, pipes, streams, message queues, and other forms of IPC) were custom-written and assigned/slaved to unique networking ports for accessing specific service daemons. Of course, the internet was a kinder, gentler place back then… and a given server may have had dozens, or even hundreds, of non-standard ports open, listening, waiting, for networking service requests. Or hacking attempts.

Today, web-services is a replacement to dedicated networking apps – all handled by your web server, and all serviced over the same network port: port 80. Since port 80 is a standard port and, usually, already open on a web-server, additional security risks by opening new non-standard ports for networking services are averted. Your web-server, such as Apache, now has the responsibility of processing the request and delivering the results to the client.

The web-services component piece is a collection of functions that you’ve written that provide remote clients access to your system. These functions are accessible only via the web services framework and while they may be duplicated from a dedicated and traditional web-based application, the web-services framework is designed as a stand-alone piece of software. Think of the web-services piece as your application’s data-processing layer minus the presentation layer. The “M” and “C” of the “MVC” model.

Initially, when I was tasked with a similar set of objectives, I initially tried xml-rpc. This led me down a rabbit-hole that spanned nearly a week of my time with the end-result being abandonment. I hit road-blocks with xml-rpc over server authentication and passing complex objects. Exacerbating the issue overall is that xml-rpc seems to be dated technology – I had a hard time locating resources that were recent.

Then I stumbled across nuSOAP – it’s free via sourceForge, stable and, using a quote from sourceForge:

“NuSOAP is a rewrite of SOAPx4, provided by NuSphere and Dietrich Ayala. It is a set of PHP classes – no PHP extensions required – that allow developers to create and consume web services based on SOAP 1.1, WSDL 1.1 and HTTP 1.0/1.1.”

nuSOAP seemed to have more of everything available: tutorials, examples, articles, blog posts. When I started my implementation with nuSOAP, the first thing I received help with was server-level authentication. I was able to immediately get my remote requests validated by the web-server and handed off to the web-services module!

The major selling point, for me, on nuSOAP is that nuSOAP is self-documenting. As part of the API functionality, nuSOAP generates HTML pages that documents the exposed services via the WSDL and also provides you with a complete XSLT definition file!

First off, download and install the nuSOAP libraries – I provided a link to the sourceForge site a couple paragraphs ago – and unpack the tarball. You’ll end-up with a directory (mine is called: ./nuSOAP) and, within that directory, is the one file you include: nusoap.php.

There are two pieces to this tutorial — a server side piece and a client-side piece. While you can execute both pieces off the same environment (machine), normally you’d use the client remotely to access the API server-side code.

What’s Not Covered:

Apache. Apache configuration for your vhost. Apache .htaccess. This article assumes that you’ve a working server and that you’re able to install and access the server files via Apache. Even if your Apache server is a localhost configuration, access the server-side files via localhost client, traversing the TCP stack locally, is still a valid method for testing your web-services server application.

Time to push up our sleeves and start working on the server code…

The Web-Services Server

Today, we’re going to write a ping server — where the server has an exposed service (method) named “ping” which takes a single argument (a string) and returns an array back to the calling client. The return array contains two associative members: a boolean (which should always be true – otherwise there are other issues…) and a string which is the modification of the original string in order to prove that, yes, we went there and we came back.

Because my project, and I’m doing this project for my new company, is going to represent significant effort, size and complexity, I’ve broken out the components of nuSOAP request into exterior files because, later, these will become control file which will, in turn, load files that have been organized into a hierarchy friendly to the application’s data model.

So, if this file, which I’ve named index.php, seems small, remember that you’re not viewing the dependent files (yet).

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25

<pre><?php
/**
*
*/

// setup...
require('./nuSOAP/nusoap.php');
// set namespace and initiate soap_server instance
$namespace = "http://myapi/index.php";
$server = new soap_server('');
// name the api
$server->configureWSDL("myapi");
// assign namespace
$server->wsdl->schemaTargetNamespace = $namespace;
// register services
require('myServiceRegistrations.php');
// load WSDL...
include('mywsdl.php');
// load services code modules...
require('myServicesModules.php');
// create HHTP listener:
$request = isset($HTTP_RAW_POST_DATA) ? $HTTP_RAW_POST_DATA : '';
$server->service($request);
exit();
?></pre>

So far, so good – let’s take a look at what we’ve just done:

we’ve included the nu_soap library…
declared our namespace (which is the URL of the api server)
instantiated a new soap_server instance and assigned it to the variable $server
initialized the WSDL
assigned the namespace variable to the WSDL
load and register our exposed services
load the WSDL
load the service code
create the HTTP listener
invoke the requested service
exit

This (index.php) file is the server-side file that will be invoked for ALL future API calls to the service. It invokes three control files which, in turn loads the services (WSDL definitions), the WSDL variable definitions (think of these as inputs and outputs to your exposed services), and the actual code for all of the services, and their supporting functions, that you’re going to expose via your API.

Side Note: This is the file you’ll reference in Apache when you’re (preferably) creating a new Virtual Host for the API. HTTP requests that resolve to your server will be serviced by Apache which will, in turn, serve the results of this program back to the client.

Next, we’re going to define the myServiceRegistrations.php file that is required by index.php. This file contains the WSDL for each and every exposed service that the API serves.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25

<pre><?php
/**
*
*
*/

$server->register('ping', array('testString' => 'xsd:string'),
array('return' => 'tns:aryReturn'),
$namespace, false, 'rpc', 'encoded',
'Summary: returns response to a ping request from the client. Response
includes testString submitted. Used to test Server response/connectivity.


Input: $testString (type: string) and random collection suffices.


Response: Service returns an array named $aryReturn with two associative
members: "status" and "data". $aryReturn["status"] should
always return true. (A time-out is an implicit false.) If no value was passed
to the service via $testString, then the value of $aryReturn["data"] will be
empty. Otherwise it will contain the string: "Rcvd: {yourString} (count)" to
show that the message was received and returned. (count) is a character count of the
passed string, also validating that the passed data was received and processed.

');</pre>

This PHP code registers a function called “ping” with the nuSOAP $server instance. The second parameter is the input to function. Note that all input (and output) parameters have to be declared as an array even if there’s only a single value being passed. Also notice that you have to type-cast the variable being passed using XML datatypes. For your data definitions, you use one of the 44 built-in datatypes defined in this document: http://www.w3.org/TR/2001/REC-xmlschema-2-20010502/.

(For more information on XSD object and XML schema, please visit: http://ws.apache.org/axis/cpp/arch/XSD_Objects.html.)

The third argument to the method “register” is the data type returned. This is a complex variable called “aryReturn” — don’t worry about right now, we’re going to define this complex-type variable in another configuration file.

The next four arguments are:

our $namespace variable we set in index.php
boolean false
‘rpc’ for the call type
‘encoded’

Use these values literally.

The last variable is a huge block of HTML. This block of HTML can be as large, or as small, as you need it to be. It’s the basis for the WSDL documentation that nuSOAP generates for your client-side developers.

When developers hit the server URL, they’ll be presented with your API documentation that nuSOAP generates from the WSDL file(s). As your API grows, exposed methods will be listed in the blue-ish box on the left side of the screen. Clicking on any of the methods will expose the details (requirements) about that method thus:

Nice, huh?

The second file (that we’ve included in our source: (mywsdl.php)) is the WSDL file that defines our data structures that are used as either inputs, outputs, or both, to the exposed services. Another thing I like about SOAP and nuSOAP is that it introduces a layer of strong-typing to the PHP stack. You can save a bit of debugging time by requiring that you call a method with EXACTLY this and return EXACTLY that. Anything else tends to generate a system-level error:

1
2
3
4

<pre>[Thu Sep 22 14:36:59 2011] [error] [client ::1] PHP Fatal error: Call to a member function addComplexType() on a non-object in /htdocs/LL2/trunk/services/mywsdl.php on line 9
[Thu Sep 22 14:36:59 2011] [error] [client ::1] PHP Stack trace:
[Thu Sep 22 14:36:59 2011] [error] [client ::1] PHP 1. {main}() /htdocs/LL2/trunk/services/ll2wsdl.php:0</pre>
<pre>

This error message, from the apache error log, is somewhat obfuscated in it's meaning. I attempted to return only the string, by itself, instead of returning the array (of two elements) that I had told nuSOAP I would return for this service. This error was generated because the types (between the code and the WSDL) of the return variable (structure) did not exactly match.

If you've only ever coded in a loosely-typed language, like PHP, than this part of SOAP is going to be a bit of a ... transition ... for you. When we say that something, be it a variable, function, or exposed service, is strongly typed, we're declaring the type of that object and, if the type of the object during run-time does not match, then SOAP will force PHP to throw a fatal as shown in the error log snippet above.

Keep this in-mind as you develop exposed services that are increasingly complex. Since the error messages tend to point you at your code, at the point of failure, it's easy to forget that that the requirements of the underpinnings (in this case, the WSDL), are the root cause of your PHP fatals.

That being said, let's take a look at the WSDL for our ping service:

1
2
3
4
5
6
7
8
9
10
11

<pre><?php
/**
* WDSL control structures
*
* initially, while in dev, this will be one large file but, later, as the product
* matures, the plan will be to break out the WSDL files into associative files based
* on the object class being defined.
*/
$server->wsdl->addComplexType('aryReturn', 'complexType', 'struct', 'all', '',
array('status' => array('name' => 'status', 'type' => 'xsd:boolean'),
'data' => array('name' => 'data', 'type' => 'xsd:string')));</pre>

We're invoking the nuSOAP method addComplexType to define a structure to the WSDL in our Table Name Space (tns). To do this, we first define the name of the structure that we're going to use: aryReturn and then we define the composition of that structure.

The declaration for this looks a lot like a standard PHP declaration for an array with the exception of the XSD (XML Schema Definition) appended at the end of each element's declaration. (See the links I embedded above for explanations and examples of valid XSD.)

XSD provides part of the strongly-typed concept for our structure elements. We're telling nuSOAP to expect a variable structure containing these named elements of this type.

What we have, then, is an associative array with two elements: 'status' and 'data'. $aryReturn['status'] and $aryReturn['data'] and they're of type BOOL and STRING respectively.

Note, finally, that this variable structure isn't confined to single-use. Once we've declared it within our tns, it's available to any exposed service where it's needed. This is the model for my common error structure -- the boolean indicates success or fail on some service operation and the data component contains the relative diagnostic information.

The third and final file we're including into the server source is the code for the exposed service. This is where you write the function handlers for your services. Since you've already defined the input parameters, and the return types for the ping service, there's very little left to do.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16

<pre><?php
/**
* ping
*
* service that confirms server availability
*
* input: any string
*
* output: reformatted string: "Rcvd: {string} (charcount{string})"
*
* @param string $testString
* @return string
*/
function ping($testString = '') {
return(array('status' => true, 'data' => 'Rcvd: ' . $testString . '(' . strlen($testString) . ')'));
}</pre>

Note the following in the code for our exposed service:

our exposed method is named ping because that's how we registered the service (myServiceRegistrations.php)
we're providing a default type-cast for the input param of string in case the service is invoked without input params.
we're returning BOOL true and prepending "Rcvd: " to the received string, and appending a character count to prove that the service successfully responded to the client's request.
the return structure exactly matches the WSDL declaration: the name of the array elements, and the element types.

If you've correctly installed and referenced (within your PHP) the nuSOAP libraries, then you should be able to load the url of the new server source file into your web browser to see the nuSOAP-generated documentation for your new web services. Click on the WSDL service function: ping to see a detailed description of the function.

If you're using IE, then clicking on the WSDL link will return the XML. If you're using Firefox, Chrome or other browsers, clicking on WSDL will display the generated XML for your service.

Now that the server is working on it's own, it remains fairly useless until we can get a client to connect to it invoke it's methods. Let's work on the client next...

The Web-Services Client

The web services client application will also be written in PHP.

The web client is an application that connects to remote server using the http port 80. To do so, you'll need the client to be aware of certain bits of information that may, or may not, be required to access the remote server.

In our client, we're not going to require remote authentication -- but I'll take a quick aside andexplain how you would include this, client-side, if your server required .htaccess authentication.

nuSOAP has a client method called setCredentials which allows you to specify your .htaccess username and password and the authentication schema. It's a single line of code which is normally used to require not only clients to login to access your API, but, once identified, you can limit the set of exposed methods available to individual clients or groups.

For example, if you have a product you've developed in-house, then you'd want full-access for your front-end web/applications servers. Your PM later decides to open a subset of the API to the general public and a subscription-based set of exposed methods in order to monetize your product. Finally, the PM also wants to "white-box" the product so that other companies can use it but with their branding and access to isolated or discrete data sets.

What you'd end-up with is several levels of client access to your web services. Implementation of limiting exposed services would be handled server-side but it would be based on your client's authentication and possibly the subject of a future tutorial...

So, to the client-side code: (name this file: apiTestClient.php)

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25

<pre><?php
// Pull in the NuSOAP code
require_once('./nuSOAP/nusoap.php');
$proxyhost = isset($_POST['proxyhost']) ? $_POST['proxyhost'] : '';
$proxyport = isset($_POST['proxyport']) ? $_POST['proxyport'] : '';
$proxyusername = isset($_POST['proxyusername']) ? $_POST['proxyusername'] : '';
$proxypassword = isset($_POST['proxypassword']) ? $_POST['proxypassword'] : '';
$useCURL = isset($_POST['usecurl']) ? $_POST['usecurl'] : '0';
$client = new nusoap_client('http://{YOURSERVERURLHERE}/index.php', false, $proxyhost, $proxyport, $proxyusername, $proxypassword);
$err = $client->getError();
if ($err) {
echo '<h2>Constructor error</h2><pre>' . $err . '</pre>';
}
$client->setUseCurl($useCURL);
$client->useHTTPPersistentConnection();
// Call the SOAP method
$result = $client->call('ping', array('testString' => 'Argle'), 'http://localhost');
// Display the result
if (!$result) {
echo "service returned false";
} else {
print_r($result);
}
unset($client);</pre>
<pre>?></pre>

The first thing we do in our client-side code is to include the nuSOAP libraries.

The next five lines of code read from the local POST environment, testing if you've established a proxy for your web-services server and, if so, populating the proxy variables.

The next line instantiates a nuSOAP client and associates it with our remote server. Change "YOURSERVERURLHERE" to the name of your apache web server URL where you have the server side code installed. (e.g.: localhost, myserver.com, etc.)

Note the name of the function call: newsoap_client()...as opposed to using the function soapclient(). This function name is legacy-compatible with PHP 5.0's instantiation call: new soapclient() - the PHP SOAP extension uses the same instantiation function name as the nuSOAP library. If you have both installed, (PHP 5.0 SOAP extension, and the nuSOAP libraries), executing the client will return errors as you've overloaded the soapclient() function. (You're calling the PHP SOAP function with the nuSOAP function parameters.) Rename the soapclient() function to the back-compatible function: newsoap_client().

The next two lines tell the nuSOAP client to useCurl, when possible and if previously saved and, if possible, to use HTTP persistent connections.

Next, we're going to consume the web-services from the server by making a call to the nuSOAP method: call(). The arguments to this method are:

the name of the service being consumed (ping)
the input string (note: all inputs must be passed as arrays!)
the namespace URI (optional: WSDL can override)

Store the results of the web-services to the aptly-named variable $results and evaluate it upon return. If the client call was not successful, then display an error message.

If the client call was successful, then display the contents of the returned array.

Note that you can run this client code from either a browser using the "file://" option or, if the client source code is accessible to apache, then you can display using a browser.

When I run this client-side software in the browser, I get displayed back:

Array ( [status] => 1 [data] => Rcvd: Argle(5) )

So, how do we know that the client went out and successfully returned from the server with the data? Simple - in the client source, you will not find the literal "Rcvd:" anywhere in the source. This data was supplied by the server-side function and returned back to the client. It's a simple example, but it provides proof-of-concept that we're successfully able to connect to a remote web-server and return data created on the remote server back to the client program.

Let's wrap this up...

Summary:

This tutorial (hopefully) explained what web-services are, and provided you with a practical example of a consumable service: ping(). Such a service would normally be invoked as a means of testing server availability.

We created a web-services server file using the nuSOAP library by defining a complex-structure (an associative array) and registering a method with the nuSOAP server. The method takes an input parameter which, although it's only a single input parameter, must be built and passed as an array construct to the server method. Next, we showed that by loading the server source into a browser, we learned the nuSOAP provides built-in documentation for your web-services structures and methods. Which is a very nice-to-have when you're creating your developer documentation! Next we created the web-services client source code file which connected to our remote server and invoked the server's method: hellow(). If all worked correctly, you displayed the string returned from the remote server within your browser window.

In the next installment, I'll cover the web-services server-side of this business and we'll see how to create the various methods for accessing a mySQL database, complex structures as input and output parameters to those methods, and general debugging techniques.

Thank-you for your patience - I hope this article helped you.

September 26, 2011

Don't miss a single post! Subscribe to my RSS feed

Press My Buttons !

Posted by : Micheal Shallop | Leave a Comment |

Part 5: Setting-up a Linux Development Machine: Virtual Hosts in Apache2

Posted by : Micheal Shallop Posted on : 21-08-2011

(No Ratings Yet)

Loading ...

Category : Technical

Tags: apache2, centos, configuration, dns, domain, host, hosts, httpd.conf, LAMP, localhost, mysql, name, nameserver, nameservice, PHP, programming, resolution, technical, tutorial, vhosts.conf, virtual, virtualhosts, web

When I am working on code project, I isolate that project within it’s own directory/repository. Further, it matters not if I’m starting a completely new project, or if I’m branching off the trunk of an existing project. As a means of imposing order over chaos, I isolate the existing project within it’s own sandbox both on the filesystem and via Apache2.

To do so requires an understanding, somewhat, of the mechanics of Apache2, DNS, and your localhost. A minimal understanding, trust me.

What it, in return, gives you is an isolated view of your code project from the web-server perspective. Cookies are isolated by domain, your document root is isolated to a single directory/repository, and you not only put your log files, just for that domain, where ever you want but you can also name them anything you want as well.

What I’ll provide you with in this installment is a rudimentary understanding of the mechanics behind virtual hosting using Apache2, a template configuration file to get you going, and the basic steps necessary to get the whole mess working. Let’s get started…

When you start a new project, if you’re checking it out from a source-code repository, you’ll typically assign it to a directory somewhere common. For example, within your home directory, you may have a folder named “code” and beneath that folder, other folders that describe either the project or the programming language you’re working in. Doesn’t really matter as the point is this: you’ve isolated your code repository from everything else on your filesystem, right?

It really doesn’t matter, to Apache2, where you create your filesystem repository. As long as the webserver pseudo-user has access permissions to the directory, then you can access the files within that directory via a web browser. The webserver has to be configured to be told that, for a given domain name, where is the documentRoot for that domain.

Some of you, at this point, may be asking: what’s a domain name and why is it important? Well, a domain name is simply a name you’ve assigned to the project to keep it separate, at least in your own head, from the other projects you may, or may not, have running on your development machine. For example, I create a new project called newWidget and it’s currently in the 1.4 revision. I’m ready to branch and write some new features for the product so, using whatever sccs tool, I branch off the trunk and create the 1.5 branch.

I check that branch out to a directory in /lampdev/php/newWidget115. I now need to do two basic things:

invent some domain name that will be used exclusively for this project and resolve the domain to my localhost
create a virtual host in apache so that apache knows that http://newW115 points to my localhost

The reasons, apart from what we’ve already discussed, is to keep your local DNS services on your local machine. If you, before entering any configuration information, entered: http://newW115 into a browser url bar, chances are very good you’re going to end-up on a search page (I’m using Chrome) or get some sort of browser error.

So the first step is to define the new domain name (again, given that we’re already checked the code out into the aforementioned directory) to the local system so that all requests to that domain are resolved locally through our name services. To do this, we’re going to sudo edit the /etc/hosts file.

This file, /etc/hosts, is the first thing checked whenever your local name services is trying to resolve a host name. If it finds a host-to-IP alias in this file, all further attempts at resolution are halted as it has successfully resolved the host name. Edit /etc/hosts to resolve your new domain. It should look something like this:

127.0.0.1 localhost codemonkey codemonkey.shallop.com codeMonkey.shallop.com newW115

The way /etc/hosts works is that you first list an IP address for the domain to resolve to - in this case, we’re using 127.0.0.1 which is TCP/IP speak for your local host. Next we list all of the domain names that are going to resolve to this IP address. In the example above, we’re resolving localhost, codemonkey, codemonkey.shallop.com, codeMonkey.shallop.com, and the new domain: newW115 all to 127.0.0.1.

Whenever I type one of these domains, for example, in to a web browser URL bar, my local host domain services won’t go out to my network name servers to resolve the domain name — it’s telling the requesting service that it’s 127.0.0.1. Note, too, that you can alias multiple domain names to the same machine.

Side Note — this is how you can blacklist certain domains from your browsing experience. Simple resolve that domain to 127.0.0.1…but that’s an article for another day…

You can also have multiple entries resolving to the same IP address. It would have been just as correct for me to have listed by /etc/hosts file as:

127.0.0.1 localhost
127.0.0.1 codemonkey
127.0.0.1 codeMonkey
127.0.0.1 codemonkey.shallop.com
127.0.0.1 codeMonkey.shallop.com
127.0.0.1 newW115

Finally, also note that a domain extension isn’t really required. We can name our domain pretty much anything we want and as long as you universally use that spelling (and case), then it will resolve locally.

Now that the domain is resolving locally, the next step is to tell Apache2 how to handle the request. When you type: http://newW115 at the browser, the browser will query local services and receive a response that the domain is handled locally. Apache2 will then say: “Oh, if it’s local, then were do I go to get the files and stuff?”

The configuration for Apache2 is done with virtual hosting. Technically, you can do this without virtual hosting — but you can only do it for one domain. If you want to locally-host multiple domains, you have to use virtual hosting.

The Apache2 configuration file lives in: /etc/httpd/conf and is named: httpd.conf. This is the main configuration file for Apache2. Some installations use a sub-directory, usually called something like: vhostsd.conf, and stores the vhosts.conf file within that directory. That’s ok, too. Apache2 is versatile that way but, for our purposes, we’re going to maintain the virtual host configuration(s) within the main conf file.

However, if you wanted to use a separate file for Virtual Hosting, all you need in your httpd.conf file is the directive:

# Virtual hosts
Include conf/extra/httpd-vhosts.conf

At the very end of httpd.conf, there’s a section called: Name-Based Virtual hosting. We’re going to append this virtual host configuration to the end of this file.

Allow me to side-step for a quick second. Consider if we were to install phpMyAdmin locally on our server because this is how we want to administer our mySQL database. We can install the program files anywhere as phpMyAdmin is just another LAMP application, right? Were we to do that, then we would need a <Directory> directive to Apache2 telling Apache2 where to look for phpMyAdmin. The domain for phpMyAdmin would still be localhost, or 127.0.0.1 or whatever else you’d defined in /etc/hosts. The location of the application can live anywhere and we’re using the conf file to tell Apache2 how to find and serve it to us when requested.

Alias /phpMyAdmin "/opt/local/www/phpmyadmin"
<Directory "/opt/local/www/phpmyadmin">
Options Indexes MultiViews
AllowOverride None
Order allow,deny
Allow from all
</Directory>

What this <Directory> directive simply does is tell Apache2 where to look for phpMyAdmin if I enter something like: http://localhost/phpMyAdmin in the URL bar of my browser. It’s not the same thing as giving phpMyAdmin it’s own domain at all.

I do this with a lot of my web applications: phpMyAdmin, mcmon, ajaxmytop, nagios, etc., simply because I don’t want to remember where the fill path name is of the applications. It’s easier to type: http://localhost/phpMyAdmin that it is to type: http://localhost/webapps/database/phpMyAdmin.

Ok, so back to domains. Here’s the template for the virtual host we’ve created in /etc/hosts: newW115:

<VirtualHost *:80>
ServerName newW115
ServerAdmin <a href="mailto:mshallop@nileguide.com">mshallop@g</a>mail.com
DocumentRoot /code/webapps/LAMP/newWidget/1-15

DirectoryIndex index.php

<Directory />
Options FollowSymLinks
AllowOverride None
</Directory>
<Directory /code/webapps/LAMP/newWidget/1-15>
Options Indexes FollowSymLinks MultiViews
AllowOverride All
Order allow,deny
allow from all
</Directory>

ScriptAlias /cgi-bin/ /usr/lib/cgi-bin/
<Directory "/usr/lib/cgi-bin">
AllowOverride None
Options +ExecCGI -MultiViews +SymLinksIfOwnerMatch
Order allow,deny
Allow from all
</Directory>

ErrorLog /var/logs/115_error.log

LogFormat "%h %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\"" combined
LogFormat "%h %l %u %t \"%r\" %>s %b" common
LogFormat "%{Referer}i -> %U" referer
LogFormat "%{User-agent}i" agent
 CustomLog /var/logs/115_log common
 ErrorLog /var/logs/115_error_log

# Possible values include: debug, info, notice, warn, error, crit,
# alert, emerg.
LogLevel warn

CustomLog /var/logs/115_access.log combined
ServerSignature On

</VirtualHost>

This is a pretty minimal configuration — but it’s the boilerplate template I use for all new domains and it works. The lines that in boldface are the lines you should change to match your environment. Note that you can pretty much put files, such as the log files, where ever you wish. I changed the names from my normal location but, as a rule, I maintain the entire environment outside of the root filesystem.

Once you’ve made your changes and saved the file, you’ll need to restart Apache2 so that it will read the new configuration. If there are errors in your configuration file, Apache2 will let you know and will refuse to start. Make sure you’ve corrected all errors and, once the server successfully restarts, you should be able to type: http://newW115 into your browser URL bar and have that domain resolve locally, and serve files from the directory you specified in the httpd.conf file.

Over time, as you add additional projects and create new code-domains, you can simply add the new <VirtualHost> directives, appending them, to the httpd.conf file as needed. When you expire and remove hosts and files, don’t forget to remove them from the Apache configuration as well.

And that’s pretty much it. This is a simple thing to set-up as we didn’t delve into anything that wasn’t plain-vanilla. For example: SSL configurations, .htacces, or the re-write engine. That’s for another day, another article.

Hope this helps…

Don't miss a single post! Subscribe to my RSS feed

Press My Buttons !

Posted by : Micheal Shallop | Leave a Comment |

Part 3: Creating Linux Development Environment (PHPStorm and the JDK)

Posted by : Micheal Shallop Posted on : 14-08-2011

(No Ratings Yet)

Loading ...

Category : Technical

Tags: apache2, centos, centos 6, desktop, development, dns, fedora, firefox, gnome, hp, ide, install, ipv6, java, jdk, jetbrains, LAMP, linux, mysql, nameserver, oracle, pavilion, phpstorm, sun

I stopped working yesterday on the installation because I hit a pothole installing PHPStorm by JetBrains.

As I mentioned in the previous article, and in case you’re just tuning in, I am first working towards a LAMP development environment on an older PC running 64-bit Linux. We’ve decided on CentOS 6 as the base distribution and installed the LAMP stack yesterday. I installed the PHPStorm package but hit a snag when I received an error message telling me that it required the JDK runtime … thingys. (Whatever – I assiduously avoid Java.)

I installed the openjdk packages with yum and got PHPStorm to start-up, albeit with many dire warnings and threats to the graphics system. Apparently PHPStorm is comfortable running only with the jdk from SUN/Oracle.

I then downloaded and RPMd the SUN/Oracle version of the jdk and restarted. What happened next were error messages telling me that I need to set-up the java (dk) environment correctly as, now, the two were conflicting with each other.

ERROR: Cannot start WebIde.
No JDK found to run WebIde. Please validate either WEBIDE_JDK, JDK_HOME or JAVA_HOME environment variable points to valid JDK installation.

See, in the linux world, the PHPStorm is launched from a shell script. It checks your environment for the JDK through these variables and, if correctly defined, launches the IDE.

There’s a java-sdk configuration file located as /etc/java/java.conf – don’t make the same mistake I made and edit this file to re-direct/create environment variables so they point to the SUN/Oracle version of the JDK.

The SUN/Oracle version of the Java SDK installed in: /usr/java/jdk1.7.0/ which will change for your system depending, I’d assume, on your distribution and version of the SDK.

To reconcile the conflicts, I used the yum installer to remove any traces of the openjdk — all packages were removed and then I did a yum clean all to reset the environment.

Since I’m the only user on this system, I next cd’d into my home directory and pulled up the .bashrc file – this will modify the bash shell environment for every terminal session I start. I added the following two lines to the .bashrc:

1 2	JDK_HOME=/usr/java/jdk1.7.0 export JDK_HOME

I exited the editor and reloaded my bash environment:

# . ./.bashrc

From there, all I need to do is start the PHPStorm shell script which launches the application and I’m good to go!

You can install the PHPStorm folder anywhere. Using your bashrc file, you can make an alias to the start-up shell script so that you can launch the IDE anywhere from the CLI environment.

alias phpstorm='nohup /home/user/folder/PhpStorm/bin/PhpStorm.sh &'

The nohup allows the program to ignore SIGHUP — in other words, if you close the terminal from where you launched PHPStorm, you will not close PHPStorm as well. The ampersand (&) at the end of the command tells the shell interpreter to launch the application as a “background” task which frees up your terminal session so that you can continue to use the shell while PHPStorm is running.

At this stage, I’m pretty much good to go for basic LAMP development. I’ve got a running mySQL server, Apache2 is good to go, and PHP5 is installed. I will enhance my environment by adding a few packages such as:

I also want to give some thought to virtual hosts — I’ll cover this topic in a future post — within my local Apache2 environment, I’m going to want to establish several different virtual host environments, each of which point to a different documentRoot location (or code repository) depending on which application/environment I’m currently working on.

I’ll also have to plan my filesystem repositories carefully — for the most part, I’ll be working as a subversion server for home-projects, while also working as a subversion client for work projects.

Which reminds me – in the first article in this series, I reported on the filesystem utilization following a clean Fedora 15 install. Here’s the state of the current filesystem (CentOS 6) following the LAMP stack install, and the install of the PHPStorm IDE and the SUN/Oracle JDK:

/ (root): 50gb, used 6%, 47gb available

/boot: 485mb, used 11%, 409mb available

/home: 864gb, used 1%, 820gb available

I’m looking pretty good for user filesystems but I’ll want to check my mySQL configuration and ensure that databases are being created in the /home filesystem and not in the /root filesystem also.

Ok – done for this weekend and off to play some Rift! Hope this helps someone!

PS: If you want some detailed tutorials on installing any of the supplemental packages I listed at the end of this article, please leave a comment!

Don't miss a single post! Subscribe to my RSS feed

Press My Buttons !

Posted by : Micheal Shallop | Leave a Comment |

Setting-up a Linux Development Client – Part 2 – CentOS 6 Install

Posted by : Micheal Shallop Posted on : 13-08-2011

(No Ratings Yet)

Loading ...

Category : Uncategorized

Tags: apache2, centos, centos 6, desktop, development, dns, fedora, fedora 15, firefox, gnome, hp, ide, install, ipv6, LAMP, linux, mysql, nameserver, pavilion, phpstorm

In the last install, I wrote about how I decided to try Fedora Linux after a nearly 10-year hiatus from the product. Unfortunately, as it turned out, my fears were not groundless and I am going to scrap the install in favor of CentOS before I get in so deep that making the switch-out becomes prohibitive.

I am going to continue to try Gnome as my desktop, however, as I did like what I saw for Gnome under Fedora. While I have always used KDE in the past, it was always accompanied by a wistful bit of: “I’ll bet the grass is greener over there…” kind of thinking. Anyone that’s ever spent anytime looking over the Gnome application offerings vs. the KDE application offerings will agree.

Time to stop wondering and start trying. I’ve downloaded the CentOS 6 x86_64 CD ISO and have booted into the desktop. It’s not nearly as polished, pretty, as the Fedora desktop — it looks more like a traditional windows set-up with the desktop icons falling down the left-side of the screen and the top-bottom menu bars. While simpler in appearance, it’s also intuitive and easier to use. Less eye-candy also means less CPU/GPU crunching resulting in improved responsiveness. (Dragging a window around in the Fedora desktop on my hardware platform was like a bit like being on a strong hallucinogenic. Or so I’ve been told.)

Anyway, I locate the “Install to Hard Drive” icon and click it…

The CentOS 6 installer opens a window in the middle of desktop (as opposed to Fedora taking over the entire desktop) and presents you with the same two start-up options: installation language and installation destination. (As I mentioned in the previous article, CentOS is a child of Fedora. I expect things to be similar. Stuff working is one such expectation.)

CentOS gives me the same options as the Fedora installer – except with less eye-candy. For example: when asking to input the root password, I’m not shown a bar indicating password strength. I just type in my password and that’s pretty much it. Also, like the previous install, I’m not going to choose the encrypted filesystem, and I’m going to go with the defaults for filesystem partitioning.

While this is installing, I’ll yak about why I’ve chosen these two distributions as my first-two choices. Ubuntu offers a great installation and configuration experience. However, after messing around with Linux distributions for 30 years, I can’t quite shake the feeling that Ubuntu in the Granimal of linux installs.

Don’t get me wrong – it’s a great install in that everything works, is highly automated, and requires little, if any, user intervention from the machine’s administrator. And that’s probably what bugs me the most about Ubuntu. As a Linux guy, I want (need) more interaction with my OS. If I was content to let me OS run off and make all the most-important decisions without asking me, I’d use Windows. Ubuntu fulfills a great niche – it introduces Windows users to Linux. I’d install Ubuntu on my Dad’s PC.

I’ve also bypassed SuSE Linux — which is surprising considering that, for nearly a decade-and-a-half, all I would consider running and installing was SuSE. This flavor of Linux, like most things German, is precise, exacting and mechanically sound. Correct, even. It’s also overbearing, heavy-handed and leaves deep footprints. The other problems that I have with SuSE is that it can be difficult to find packages tailored for it’s installation base. While SuSE enjoys a wide-variety of software, there always seems to be those few-dozen packages you want to install but can’t locate the ports to the SuSE distribution. In that, it’s like the Dewey (Malcolm in the Middle reference) of Linux installs: unprepossessing and brilliant but relatively scarce when it comes to applicable resourcing.

I’ve never been a big fan of Debian simply because they move in geological-timeframes when it comes to engineering releases. Oh, look, kernel 2.26.9999 is out! (Debian: happy with 2.123, thank you.) Geh. What it lacks in contemporary packaging, it more than makes up with in stability. I, on the other hand, tend to blow through distributions like the end is near so Debian isn’t really for me.

I tried Mandriva once and, as a result, got sucked into this weird mail hell back when I was running my own DNS and MX servers. I really tried to make it work but it just got too … weird for me. It may have improved in recent years but I’ve never had enough of it catch my eye to really care enough to revisit it.

Rebooting the CentOS 6 Live CD was better than the Fedora Live CD as CentOS actually gave me a ‘reboot’ option whereas Fedora would only let me ‘suspend’…whatever that means…

I configured the user and the network time and then was presented with an alert: “Insufficient Memory to Start kdump” … which made me think I had crashed the install…turns out, it was just telling me I couldn’t start the monitor itself.

On to the login…

Well, CentOS 6 is definitely a derivative of Fedora 15. Although the desktop is radically different, the first thing I try is FireFox — and am immediately told that I can’t access any off-site web page. Although I can ping and resolve hosts from terminal, FireFox cannot do so from the browser. So the same crappy DNS issue which plagues Fedora was inherited by CentOS. Great. Starting to get an idea of where all this is eventually going to end up…

The network configuration applet in CentOS allows me to edit and add google’s nameserver and things start to work in the browser immediately thereafter. For some reason, I wasn’t able to get this to work in Fedora so, bonus. Also, my screen resolution is at the highest at 1280 x 1024 and that gives me a happy, too.

I start the software update and am informed that all my software is currently up-to-date and I do not need to additional software. That strikes me more as a software fail…so I run yum update from the command line as root (side note: either I didn’t see the option to create my new user as an admin, or it didn’t exist, but regardless, I can’t sudo…) and I’m suddenly off-and-installing 237 total packages… so, clearly something in the GUI version of the software update failed and now I’m thinking that, because I didn’t have sudo privileges, it was my account exec’ing the command.

CentOS 6 will allow you to login graphically as root. And thereafter puts so many scare-ware pops on the screen that you eventually, submissively, quietly and quickly edit the sudousers file and logout. Now that my main account has sudo access, I never need to hit root again.

Quick download and now Chrome is my default browser…time to try to install some development tools…

The first package I’m going to install, from the Add/Remove Software package manager, is the MySQL server and related files package which is an 8.1mb download…I have to also install dependent packages for perl support and client programs and shared libs, which is ok…PHP 5.3.2 is the next item to be installed and I install all packages except for postgres.

At this point, I have a LAMP stack installed, but it’s not running… starting off with mysql:

# sudo chkconfig --level 2345 mysqld on

# sudo /etc/init.d/mysqld start

# mysql -uroot

mysql> use mysql;

mysql> update mysql set password=password('yourPasswordHere') where user='root';

mysql> exit;

This set of commands sets-up mysql to run at start time (run levels 2, 3, 4, and 5) and then starts the mysql server. Next, you invoke mysql as root and reset the root password to something other than the default, which is nothing.

–> mySQL is now running.

For Apache, we’re going to leave virtual hosts alone for a future article, and just make sure that the webserver will execute at boot, and that we can serve system information…

# sudo chkconfig --level 2345 httpd on

# sudo /usr/sbin/apachectl start

If you ps -ef | grep httpd you’ll see a list of the running apache servers…you can also open up http://localhost in a browser window and you should see the CentOS Apache 2 Test Page. Now we have to confirm that we have PHP installed and running, along with a few other modules. By default, your web server DocumentRoot is in /var/www/html. Using the terminal, cd into this directory and type the following:

# sudo vi snitch.php

i

<?php

phpinfo();

<esc>:wq

This creates a little snitch file in your DocumentRoot which you can load in a browser — it then dumps your LAMP configuration to your browser window. At the very top of the display, it should tell you what version of PHP you’re running. (Mine reports version 5.3.2.) Important to me, at this stage, is that I have memcache, soap, mysql, and ODBC drivers installed.

The last stage for me is to install my IDE. I own a license for JetBrains PHPStorm which I personally prefer. It’s not freeware but if you can afford the license costs, it’s probably the best IDE you can get for the price. I use it on all environments (Mac, Windows and Linux). I also noticed that you can install the Eclipse IDE using the software installer — this is very similar to PHPStorm.

To get PHPStorm up and running, I need the SUN/Oracle version of the JDK — not the openJDK. I did get it running, but not without DIRE and URGENT messages prophesying the END OF THE WORLD, or at least my video display, should I continue. Point is, I did get it installed, configured, licensed. Then I de-installed the openJDK and went hunting for the SUN/Oracle JDK.

Which will be covered in the next installment…

Don't miss a single post! Subscribe to my RSS feed

Press My Buttons !

Posted by : Micheal Shallop | Leave a Comment |

Installing a Linux-Based Development Environment – Fedora 15 Fails

Posted by : Micheal Shallop Posted on : 13-08-2011

(No Ratings Yet)

Loading ...

Category : Technical

Tags: desktop, development, dns, fedora, fedora 15, firefox, gnome, google, hp, ide, install, LAMP, linux, nameserver, pavilion, tutorial, web

I’ve scrapped my VOIP project with the realization that, advanced DSL or not (whatever THAT is, AT&T), I just don’t have the upload bandwidth to support the project to a place of anything usable.

So, after a couple months of looking at an inactive PC sitting on the top of my desk doing absolutely nothing, except irritating my OCD bone, I decided to embark on a new project — installing a linux-desktop based IDE platform.

Keeping my goals simple, I want a LAMP development environment with the option to later build out to a compiled-based environment/IDE for C/C++.

I am installing on an older PC – an HP Pavilion Slimline S3300f that initially had Windows Vista installed but has since advanced and evolved to host a variety of various Linux platforms. The CPU is a AMD Athalon X2 64-bit Dual-Core hosting 2-gb of PC2-5300 SDRAM, a 500-gb hard drive and a 2007 Phoenix BIOS. Perfect for Linux.

I thought about which flavor of Linux to install – over the years I’ve used pretty much all of them and finally decided to try something new…new in that I’ve not used it in quite a while. I decided on the latest 64-bit Fedora install with the Gnome IDE available from the Fedora Project. While I normally choose CentOS as my linux-of-choice, I picked Fedora because I wanted to re-visit the distribution.

I’ve not used Fedora since 2002. I was never that enthusiastic about Fedora since it seemed, to me at the time, to spend too much time customizing what was, at that time, standard unix configurations. Well, I really like CentOS, which is based on Fedora, so maybe it’s time to put past-prejudices aside and see what’s up with one of the most popular distributions in the world.

And, while we’re at it, let’s eliminate our past-dependencies on KDE and stagger completely free from our comfort zone and go with the Gnome desktop, shall we? <muted screaming noises>

I downloaded the Fedora x86_64 CD ISO from the Fedora project and booted my Pavilion off the CD. A desktop soon appeared which, I have to admit, is really nice. The default boot brings you into a run-time environment chock-full of neat applications which I didn’t take the time to explore…unfortunately…but I do like the concept. This ISO gives you a portable and secure desktop system that, with internet access, provides you with the ultimate, secure, thin-client.

Anyway, I clicked on the install-to-hard-drive option and dove into the Fedora installation program. I have no screen shots for you - I am writing this blog on my Mac OS-X bootmaster partition of Windows 7 (Ultimate-64) because I’m too lazy to reboot back into Mac OS X and I’m going to play Rift later…

The Fedora installer took two options, initially — my language and the storage device choice. English and local (as, say opposed to SAN) and it paused because I had encrypted, in a previous install, the hard drive. So, it waiting patiently until I was able to dredge the password out of my abused long-term memory. Luckily, I had saved it in a keychain…

I then configured standard stuff – machine name (codemonkey), root passwords, and storage option. I elected to use the entire hard-drive which will wipe whatever crap I previously had installed/stored. In this, I was also given the option to encrypt the file system (which is cool in a James Bond sort of way but a pain-in-the-ass in RL because you have to remember your password and you do take a CPU hit), and to review the partitioning schema. I went with no encryption and no review — nuke it, Chuck.

While it’s installing, I wanted to add that this system is using a 17″ LCD monitor. While this type of monitor used to be the shizzle, it’s now dwarfed by my iMac’s 27″ display and the slaved 23″ LCD next to that. On the other hand, I now have 71″ of LCD wrapping around my two keyboards…which is pretty cool. Traditionally, Linux never seemed to give me the same video resolutions on this monitor that Windows was able to. 8×6 or even 1024×768 just doesn’t seem to cut it anymore, so I’m curious as to what Redhat will deliver.

The Fedora installer copied the live image to my hard drive fairly quickly. I noticed that the installer should update it’s (C) date and include 2011 – I believe this install was last update just a few days ago…and, the entire time the installer is running as a windowed app beneath the live desktop. My first linux install was sometime around 1991 under version 0.91 or so — so the installers have definitely evolved!

After about 10 minutes, I was presented with the reboot message….so I popped the CD out of the tray and rebooted…exiting the installer took me back to the live desktop. I had to manually power off the machine, reboot, remove the CD from the tray…so I missed the short window to pop back into the BIOS to reset the boot order.

Booting from the hard drive, I was brought to a Welcome screen (very snazzy!) and querying me for more information. I was asked to create a user account – which I added to the administrator’s group assuming that this would bestow me with su privileges. I then set-up the NTP server and submitted my hardware profile back to the Fedora Project.

Following this, I was presented with the log-in window…and I after entering in my password, I was introduced to my first-ever Gnome desktop environment under Fedora. (2001 Space Odyssey music playing in my head.)

I’m immediately happy because I’m at 1280×1024 resolution for the first time, ever, in a linux install on this hardware. I’m going to play around with my desktop for a bit and get used to the UX while finding a comfortable configuration.

The system information informs me that I am running Gnome 3.0.1 with 1.8gb of RAM, and AMD Athalon 64×2 Dual Core Processor 5000+ X2, a Gallium 0.4 on NV4C graphic driver, 64-bit OS and 931.4 gb free on the hard drive. df -h output shows that I’ve used 6% of my 50gb root partition and I’m worrying that this won’t be enough to support system updates. Looks like everything will have to be installed into /home.

So, I have the following (default) filesystem set-up:

/ (root) at 50gb with 94% free
/boot at 500mb with 91% free
/home at 864gb with 99% free
/tmp at 50gb with 94% free

Interesting that this window also provides me with update notification which was my next step…installation of updates and installation of new packages…the software has located a staggering 634 updates at 197.3mb…the majority of which appear to be bug-fixes with the remaining packages listed as enhancements.

The package install fails immediately with what I suspect are permissions issues. I then next learn that Fedora has disabled root logins into Gnome. Since this is an administrator account, I need to figure out how to install these updates. I re-login and try to re-install and get another failure, albeit a different failure message. I try to, and succeed, in installing a single update and the installer informs me I have to re-login for the updates to be effective. I do so, but it’s a bit of an Easter-Egg-hunt as the installer reset my terminal…I have to Ctrl-Alt-F8 to get back to my terminal session and I re-login and restart the software update program.

And I keep getting errors…geh… time for terminal. I google the issue and discover that it’s a known bug in the package kit installer and to use yum instead. I switch to terminal, su to root and yum update … The (now) 302 updates I need spin down to the box effortlessly and the entire system is updated in about 10 minutes.

The next step is to identify and install the development components. Also, I’ll need to install some server software (apache2, php5, mysql, mongodb, etc.) for the LAMP development environment. For now, what I have is a working desktop environment.

I pulled out an old set of Logitech speakers from the garage, rescuing them from an in-progress garage-sale, and now I have working sound which is cool. I also want to get network shares up and running so that I can copy files back-and-forth across the network.

As a final note to this post, I had a hell of a time trying to get Firefox to connect. It’s a DNS server issue with Fedora 15 and FF that’s based on auto-enabling of IPV6 protocols. To “fix” the issue, simply add google’s default nameserver configuration to your /etc/resolv.conf file:

nameserver 8.8.8.8

And any name-server resolution issues in firefox should disappear immediately.

[EDIT]

I’m going to have to trash Fedora 15 as a viable option. I had the desktop lock-up on me while displaying the application listings following a Chrome install. When I rebooted, I lost the nameserver configuration that I added to /etc/resolv.conf (above) and had to manually add it so that other network-based applications (browser, chat client) would work.

All in all, there’s probably fixes for these issue but at this point I really don’t care. There are other Linux distributions out there that are stable and don’t distract you from accomplishing real work with niggling little issues such as what I’ve experienced. The application lock-up was unforgivable, to be honest. If I wanted to experience fun like that, I’d install Vista back on this machine.

I’m really kind of surprised to have this much trouble with a Fedora install considering their reputation for reliability and stability in the enterprise. Clearly, Fedora isn’t ready for a desktop/client market.

I’m in-process of downloading CentOS 6 — we’ll see how that goes in the next installation….

Don't miss a single post! Subscribe to my RSS feed

Press My Buttons !

Posted by : Micheal Shallop | Leave a Comment |

Featured Posts

Subscribe

Testing Arrays in PHP – Back to Basics…

PHP: Comparing Object Structures

Why is my mongo query so slow?

Searching MongoDB Sub-Documents…

mongodb.findOne() — calling with PHP variables (not literals)

Web Services with PHP & nuSoap – Part 1.1

Introduction

The Web-Services Server

The Web-Services Client

Summary:

Part 5: Setting-up a Linux Development Machine: Virtual Hosts in Apache2

Part 3: Creating Linux Development Environment (PHPStorm and the JDK)

Setting-up a Linux Development Client – Part 2 – CentOS 6 Install

Installing a Linux-Based Development Environment – Fedora 15 Fails

Login

Pages

San Ysido Weather

Recent Twitage

May 2013
S	M	T	W	T	F	S
« Jan
			1	2	3	4
5	6	7	8	9	10	11
12	13	14	15	16	17	18
19	20	21	22	23	24	25
26	27	28	29	30	31