Blog

Blog Archives - 14 Record(s)

Year: '2015' - Month: '2'

26
February 2015

Gavin Pickin

HTML 5 Search Input with jQuery Multi Token Search

Javascript, jQuery

Recently I found myself playing with another HTML 5 form element, the Search Input. Of course the form field support is not there yet… so I decided to use jQuery’s version since I was already using jQuery Mobile on an App. Quite a nice upgrade, it gives you some standard features that you might normally struggle to complete on your own… like the rounded corners, and the cancel/clear search field. Of course webkit goes further with remembering the last x searches, and can even fire a “search” event when a user pauses while typing… but sadly, we have to live in a world where you can’t use all the bells and whistles all at once… but jQuery as usual fills in the important gaps nicely.

As I mentioned, working on this jQuery Mobile App (I started this one before I spent much time working with Ionic which is amazing if you didn’t know already) I wanted to use a Search Input, to filter some tables of data. jQuery offers some simple methods which I have used before, but I thought I’d try out this new input type, search.

Assuming the “search” event is not safe, I added the keyup event handler.

<!--- HTML --->
<div data-role="fieldcontain">
    <label for="training_filter_field">Filter Trainings:</label>
    <input type="search" name="training_filter_field" id="training_filter_field" value="" />
</div>

 

//JS On Document Ready
var $rows = $('#table tr');
$('#training_filter_field').keyup(function() {
    var val = $.trim($(this).val()).replace(/ +/g, ' ').toLowerCase();
    $rows.show().filter(function() {
        var text = $(this).text().replace(/\s+/g, ' ').toLowerCase();
        return !~text.indexOf(val);
    }).hide();
});

That got me thinking, what about the Search Cancel Button? What events does that emit? Of course the beauty of the input element is it automatically clears the search input, but we need to reset the search when someone clicks it. After doing some research, I did not find a clear answer, so I decided to experiment. I found that because the focus on the search input was lost and then regained as the field was reset, it fires the change event on the text input. To not duplicate the search code, I just use the on change event handler to trigger a keyup event on the search field with this code.

$('#training_filter_field').on('change', function(){
     $('#training_filter_field').trigger('keyup');
});

That seemed to work perfectly.

Short and sweet, and a power small filter… but I wanted more. I wondered if we could search multiple tokens, to create a more powerful filter, or would I have to resort to building my own. Coming from search side, I am used to building the UI to do whatever I need it to, but I wanted to see what was out there already.
Sure enough, in a few clicks, with the right search phrase, I had found a great example, simple example, like above ( it was so good I replaced my simple code above with this ) and then a slick multiple token filter.

Thank you dfsq from Stackoverflow

 

var $rows = $('#training-list tbody tr');
$('#training_filter_field').keyup(function() {
    var val = '^(?=.*\\b' + $.trim($(this).val()).split(/\s+/).join('\\b)(?=.*\\b') + ').*$',
        reg = RegExp(val, 'i'),
        text;

    $rows.show().filter(function() {
        text = $(this).text().replace(/\s+/g, ' ');
        return !reg.test(text);
    }).hide();
});

$('#training_filter_field').on('change', function(){
    $('#training_filter_field').trigger('keyup');
});

With that small chunk of code, with the on change search input, I had a nice simple but powerful UI for search and filter on my table.
Of course, some of you might say that you don’t need jquery for everything, and you’re right. I know this can be done without jQuery, but its baked into my app, and this can’t get much easier, so why not make use of it.

Gotta love jQuery, new standard features coming like HTML 5 forms, and even StackOverflow. Although I have come to learn that StackOverflow is a lot like an antique shop, there is a lot of crap in there, but there are some priceless items hiding there too.
 

20
February 2015

Gavin Pickin

CommandBox Roadshow - Episode 3 - Building Custom Commands - with ME

Apache, CFML Language, CFML Server, CommandBox, Lucee, Online Interactive Learning

I have been playing with Ortus Solution's amazing new tool CommandBox a lot lately. After a couple of posts on my blog, Brad Wood from Ortus Solutions reached out to me and asked if I would be interested in sharing what I have been doing with the community. A lot of time and money was invested by Ortus Solutions to make this product, and every knows ColdFusion / CFML in general needed something like this for a long time now, so of course I accepted their offer. I have already posted a couple of blog posts on the Ortus blog (links below), and today we recorded an episode of the CommandBox Roadshow, with yours truly in charge of the Adobe Connect session.

It was one of those mornings so we didn't get the blog posts, emails and tweets out there in time to get a big audience, but we recorded the session, so please check it out, the link is below.

 

I walked through some of the features of my kiwiSays command, and even I learned a couple of things from Brad while recording. I didn't know that the "r" command reloaded CommandBox, so as you develop, you can quickly reload, so you can test as you go. I was closing my terminal and reopening it, which isn't efficient, but after working with Cordova, and XCode, and IOS lately, I am used to not being efficient :)

I mentioned the "CommandBox - Commands can be clean code too" post from Ortus where I took my monolith Run function from the addWebsite command, and made it "CLEAN" by refactoring into smaller functions. We looked at how you can have namespaces (or groups of commands) and how you can save settings in a json object, and how one command can call other functions.

I also walked you through how to build your own command, and in the BOX spirit, I called this one BoximusPrimeSays, If you want to open your default browser, commandbox comes with a browse command already, and that is pretty slick. I wanted to go further and open different browsers. So we made a couple of smaller commands, to pass a url to Firefox, or Chrome, or Safari. To make it really useful, we made a "boximusPrimeSays open http://www.gpickin.com" command, which opens MULTIPLE browsers all to the same url, from a simple command. I hope you take the time to make a command yourself, and see how easy and powerful CommandBox is.

CommandBox is a great Package Manager, and REPL, but Commands make it a tool that we all can use... hope you make some and share them with the community too.

18
February 2015

Gavin Pickin

Packt Publishing Encourages Customers to Learn New Skills with 18 free e-books

Books and Training, Chit Chat, Steals and Deals

Develop new skills and unlock valuable knowledge with a Free Ebook Every DayAs an avid techie, and a habitual learner, I love ebook publishers, and Packt Publishing is one of the top ones on my list. I have bought many a book, hard copy and e-book over time from Packt Publishing, and have mentioned that in my blog in a previous post Packt Publishing $5 Ebook Special and Review of Object Oriented Programming for ColdFusion and Ebook Sale Ending . Today I'm writing this post to inform you of a great Promotion Packt Publishing has started, which includes 18 days of Free learning, with a different e-book available, for free, every day from Feb 16th, through March 5th.

Here is the copy from their Press Release

Packt Publishing is encouraging customers to develop new skills and try new technologies with 18 days of Free Learning. Beginning on Monday 16th February, the publisher is inviting customers to claim a free eBook every day to learn a new skill and to get to know the wide range of technologies and subjects that can be found across their extensive library of titles.

Each eBook will only be available for free for 24 hours – so make sure you visit Packt’s website every day from the 16th February to March 5th to grab your daily Free Learning fix. With such a wide range of topics tipped to potentially feature – from Drupal to Learning Play, Magento to Moodle, Selenium to OpenLayers, Maya Programming to Linux Shell Scripting, Redmine to BackTrack - be sure to take this opportunity to try something new.

All customers have to do is simply click on the day’s free eBook and it will instantly be added to their account. New customers are also encouraged to take advantage, with the offer being a great opportunity to get to know Packt a little better – all that’s required is a Packt account.

You can view today's Free Book from Packt Publishing by visiting this link https://www.packtpub.com/packt/offers/free-learning#

Check out their selection while you are there.
I always try to support those, who support others, and with this free promotion, on top of their $5 e-book deals, and other great offers, including subscription plans, Packt Publishing is doing great things for our industry.

So what are you waiting for, click click click.

17
February 2015

Gavin Pickin

WebSQL - When is a Javascript Object not a Javascript Object?

Cordova / Phonegap, Javascript, Mobile Development, Techie Gotchas, WebSQL

After a short break from WebSQL, I am back to talk to you about WebSQL again, as I continue my work on my Cordova / Phonegap cross platform app. Today I ran into one of those Gotchas, that cost me a couple hours of time trying to debug a strange happening in my app. It is one of those things in our developers lives, if you only knew what you didn’t know, you'd know more than you know now. In finding this gotcha, I remembered a couple of other lessons when using Web Developer tools, which you may or may not know… so I’ll share those too.

The context of the situation made it worse than it would have been normally, so I’ll skip most of it… and get down to the nuts and bolts. When you use WebSQL and create run a query, like below, you are returned a SQLResultSet, which has rows of items.

app.db.readTransaction(function (tx) {
          tx.executeSql('SELECT * FROM people',
               [],
               function( tx, peopleList ){ doSomethingWithPeopleList( peopleList ); },
                              errorCall
               );
          });
     });

 

You can access the first item in the Result Set by the following syntax.

var person = peopleList.rows.item(0); // from 0 to peopleList.rows.length-1

 

If you console.log( person ), you see a Javascript Object… which is really handy to work with, dot notation for property access, not too much reinventing the wheel… or so I thought.

My issue came up where I was passing this object reference through several functions, updating properties as necessary, and then passing the object back to be saved if the object was dirty. My problem was, it was never dirty… and therefore never updating.

First step was to console.log the object, at several steps in the process. Even before and after setting a property, the objects were identical. I decided to add a new property, and see if that made a difference. When I looked, all of my objects had the new property, but the old was unchanged. I started to think about async and possible conflicts, and thought I was losing my mind, when something triggered a memory. When using Web Tools to debug, and you console.log a javascript object, console.log uses references, so if you console.log it 3-4 times, when you view the log, they all look the same, because its a ref to the object, and how that object looked at last usage.

There is another console method, which works (usually in most dev tools, I have not tested them all) where you console.dir( person ) instead of console.log( person ) and it actually dumps the object at that point in time, and not just a reference. Good to know, and good to remember. I wonder how many of you have battled that one before.

Back to the main problem, why were some of the properties read only, but others were not. I even compared the same object as is, and then tried a JSON.parse( JSON.stringify( person )) and compared them… visually there was no difference. At this point, I finally found the magic words in google ( previous searches did not turn anything up ) and found a couple of posts on stack overflow. The tldr; WebSQL result sets in some implementations do indeed output an object with immutable properties.

http://stackoverflow.com/questions/19999906/websql-are-returned-rows-in-sqlresultsetrowlist-immutable

http://stackoverflow.com/questions/5093870/immutable-chrome-sqlite-return-objects

In short, if you are working with WebSQL results, the items returned are Javascript Objects (the title might be link bait, maybe), but it is quite likely, that they are read only Javascript Objects. There were several suggestions on how to change that, with Object.defineProperty and jQuery.extend() and _.extend().. and of course, the JSON.stringify and JSON.parse functions which convert the object into a string, and back again.

I am not sure if having the data as immutable or not is a good idea, it does make converting to use with localStorage to WebSQL more complicated. If you did convert from localStorage to WebSQL you could mask that in the DAO, although that seems to be a lot of work in preparation alone.

Having an immutable set of properties might be nice, to be able to add additional fields, so you could easily compare original and updated fields.
Knowing this, I can work around it, but it wasn’t easy to determine in the first place. In doing so, I remembered a key debugging tip with console.log and value vs references, and console.dir.

Hopefully others can learn from my tough learned lesson.

13
February 2015

Gavin Pickin

How to get your Tomcat to pounce on startup, not crawl

Apache, CFML Server, Server Admin

As promised in my last few posts, I am finally going to give you a couple of Tomcat tips and tricks, that can seriously speed up your Tomcat startup time. In all fairness, they are simple configurations, and I am not the first to find them, not even the first CFMLer to find them. This is how I got my Tomcat startup from 164 seconds, down to 8 seconds, a gain of 156 seconds, in 1 line of code.

The first thing you have to remember, whether we are running Lucee, or Railo, we are running on the JVM. Tomcat is a Java Servlet and it is much more powerful than our lil old cfml engines. Knowing this, Tomcat has a few settings in given files that don’t apply to MOST of our CFML servers… the big one, is the JAR scanning.
Note: There could be an odd chance this might affect some weird setup, I’m just not sure what at this point.

"The Jar Scanner element represents the component that is used to scan the web application for JAR files and directories of class files. It is typically used during web application start to identify configuration files such as TLDs or web-fragment.xml files that must be processed as part of the web application initialization."

Taken from the Apache Tomcat 8 Configuration Referencehttp://tomcat.apache.org/tomcat-8.0-doc/config/jar-scanner.html

For every host / context, tomcat is scanning every folder it knows about (slight exaggeration), and you can see in the logs that it fails to find TLDs and configuration information, that it warns you that you could save time.

org.apache.jasper.servlet.TldScanner.scanJars At least one JAR was scanned for TLDs yet contained no TLDs. Enable debug logging for this logger for a complete list of JARs that were scanned but no TLDs were found in them. Skipping unneeded JARs during scanning can improve startup time and JSP compilation time.

So we are wasting time here, great, how do we fix it. There are two solutions.

// File: conf/context.xml

 

Inside the Context element, you have watched resources, but you can also set JarScanner with some params. By default scanAllDirectories, scanAllFiles, and scanBootstrapClassPath are set to false, but scanClassPath is true. Setting this, will set that to false too.

<JarScanner scanClassPath="false" />

 

Restart Tomcat, and you should see the startup time disappear.
My dev machine right now, has only a few Host setup in my Server.xml, since i have been experimenting with lots of alternatives to using manually coded Hosts, but even with that, my startup times looked like this.

12-Feb-2015 21:43:39.592 INFO [main] org.apache.catalina.startup.Catalina.start Server startup in 23576 ms
// Versus 
12-Feb-2015 21:48:40.840 INFO [main] org.apache.catalina.startup.Catalina.start Server startup in 3287 ms

 

Now, if you are concerned, and you want some of your JARs to be scanned, the 2nd option will be better for you.
Note, I am referring to Tomcat 8, Tomcat 7 had 3 settings in the catalina.properties, not just 1.

// File: conf/catalina.properties approx line 96 for standard Lucee Express

tomcat.util.scan.StandardJarScanFilter.jarsToSkip= \
bootstrap.jar,commons-daemon.jar,tomcat-juli.jar,\
annotations-api.jar,el-api.jar,jsp-api.jar,servlet-api.jar,websocket-api.jar,\
catalina.jar,catalina-ant.jar,catalina-ha.jar,catalina-storeconfig.jar,\
catalina-tribes.jar,\
jasper.jar,jasper-el.jar,ecj-*.jar,\
tomcat-api.jar,tomcat-util.jar,tomcat-util-scan.jar,tomcat-coyote.jar,\
tomcat-dbcp.jar,tomcat-jni.jar,tomcat-spdy.jar,tomcat-websocket.jar,\
tomcat-i18n-en.jar,tomcat-i18n-es.jar,tomcat-i18n-fr.jar,tomcat-i18n-ja.jar,\
tomcat-juli-adapters.jar,catalina-jmx-remote.jar,catalina-ws.jar,\
tomcat-jdbc.jar,\
tools.jar,\
commons-beanutils*.jar,commons-codec*.jar,commons-collections*.jar,\
commons-dbcp*.jar,commons-digester*.jar,commons-fileupload*.jar,\
commons-httpclient*.jar,commons-io*.jar,commons-lang*.jar,commons-logging*.jar,\
commons-math*.jar,commons-pool*.jar,\
jstl.jar,\
geronimo-spec-jaxrpc*.jar,wsdl4j*.jar,\
ant.jar,ant-junit*.jar,aspectj*.jar,jmx.jar,h2*.jar,hibernate*.jar,httpclient*.jar,\
jmx-tools.jar,jta*.jar,log4j*.jar,mail*.jar,slf4j*.jar,\
xercesImpl.jar,xmlParserAPIs.jar,xml-apis.jar,\
junit.jar,junit-*.jar,ant-launcher.jar,\
cobertura-*.jar,asm-*.jar,dom4j-*.jar,icu4j-*.jar,jaxen-*.jar,jdom-*.jar,\
jetty-*.jar,oro-*.jar,servlet-api-*.jar,tagsoup-*.jar,xmlParserAPIs-*.jar,\
xom-*.jar

 

This property lists all of the jars to skip… so if you want, you can name all of the jars to skip, but it extensive, and daunting.
Or you can do the equivalent of the last setting, and set this line to

tomcat.util.scan.StandardJarScanFilter.jarsToSkip=*

 

This will skip all JARS, and you will see almost identical startup.
That is all of the magic, its 1 setting, and it saves about 5-6 seconds per Host on each and every startup. I previous had 20-30 sites and was waiting for up to 5 minutes, which is ridiculous. Especially considering we do not need scanning.

As I mentioned earlier, Tony Junkes was looking into this when he first moved to Tomcat 8, and posted his article on Fixing Slow Railo Startup on Tomcat 8
Tony goes into more details on this findings, plus a few other bugs he found, so I will not repeat that, please go read his article.

At the same time we posted that, I was still running Tomcat 7 and round that there were two other settings, shown below, but the best performance increase was the same setting still available in Tomcat 8

org.apache.catalina.startup.ContextConfig.jarsToSkip=* = 150 second gain
org.apache.catalina.startup.TldConfig.jarsToSkip=*     = No real gain
tomcat.util.scan.DefaultJarScanner.jarsToSkip=*  156 second gain

Although this post is aimed at Tomcat 8, if you are still on Tomcat 7, it still might be fruitful to test this out.

If anyone knows why you might need to leave this on, in our normal CFML running servers, please share that information.

I hope it was worth the wait, go try it.
Please give some feedback in the comments to the BEFORE and AFTER times, and the number of Host records if you are up to counting them, please.

My dev server before the Host cutting was 95% savings on 25 hosts
Current dev server, it was 86.1% savings with 3 hosts.

12
February 2015

Gavin Pickin

Pros and Cons of Mod_CFML with Tomcat for Lucee

Apache, CFML Server, Lucee, Migrating to Railo, Server Admin, Tools and IDEs

I have been talking a lot lately about different configuration options, since Lucee is the hot new thing since ‘POCKETS’. Everyone has been asking about best ways to work with Tomcat, versus other servlet containers, and so I have been playing… and have released a few posts on the matter. Every time I post anything on Tomcat, I always get asked about MOD_CFML, so I thought it was about time I wrote a post on what I knew about it, as always, I am not the expert, I am just sharing some of my knowledge, hopefully others will share too, and we can all learn.

What is Mod_CFML?

Mod_CFML is a connector tool to help IIS and Apache communicate with Tomcat and eliminate some of the additional overhead for Server Admins, by using a Tomcat Valve. It essentially passes information from Apache to Tomcat the first time a new “WEBSITE” is accessed since the last time Tomcat (which runs Railo/Lucee) was started up. It passed through the Server Name and the WebsiteRoot so Tomcat can setup a Virtual Host, similar to how Apache has its own Virtual Hosts. Mod_CFML is a community driven project, and I know Vivio’s Jordan Michael’s name is in the mod_cfml.pm perl file, and he is very helpful regarding this subject on the google groups in Lucee, and previously with Railo.

Why do we need something like Mod_CFML?

One of the biggest issues we have with using Railo and now Lucee, has been the way it connects to Tomcat. In the good old days, you would install ColdFusion and whatever folder you setup IIS or your web server of choice, the web server would handle static content, and the ColdFusion engine would just work with the dynamic content. These days with Tomcat, being its own fully fledged Web Server, it isn’t as simple to setup, especially if you want something like Apache to still do the front, we have to PROXY that information through to Tomcat, so it can take care of the CFML requests, and send the info back to Apache (or IIS). Tomcat needs certain information, so it can be setup and ready to handle those requests, so we have several approaches to solve this problem, this is one of them.

Tomcat is known to be VERY SLOW on startup… ridiculously slow, and the most Hosts and Context, the more unpacking work that needs to be done, the worse it gets. Anything you can do to save yourself a restart of Tomcat was always a HUGE plus.

I have some information on speeding up a Tomcat 7 and Tomcat 8 startup, which will help most of us who use Tomcat for CFML engines only. I got my Tomcat startup down from 60 seconds to 3 seconds, with only 10 sites. With 50 sites, you could be waiting for 5-6 minutes normally, but we can reduce the startup to only a few seconds. I will be posting that very soon, so please check back for that blog post. Tony Junkes also released some information on speeding up your Tomcat 8 server too.

What are some of the other solutions?

  • We can manually create a HOST element in the Server.xml of Tomcat, and restart Tomcat.
     
  • We can setup an XML INCLUDE to hold manually created HOST elements which is included in Server.xml each time Tomcat is restarted. I have posted a few blogs on this approach, if you are doing manually created host elements, this is much easier than dealing with the XML directly. Read this post called Adding Goodies to Server.xml to make life easier with Lucee.
     
  • We can use a Catchall HOST in the Tomcat Server.xml and by passing relative path information in the ProxyPassMatch, Tomcat can serve the files back to Apache, with minimal difference. This is a solid practice for dev, but all of your sites are sharing one Context which is not recommended by J2EE standards, and it does affect the CGI Environment Variables. Although its not too difficult to reset those in your Application.cfc, with a / mapping for example. I wrote about this approach just a few days ago, one of the big pluses, is you do not have to restart Tomcat. Read Apache and Tomcat - Save yourself the XML Editing - No more Tomcat restarts

Why should I consider using Mod_CFML?

  • It creates the HOST record on the fly, you do not need to create it manually, as soon as a request to an Apache site is made, that proxies the request over to Tomcat, using the Valve setup, it will setup a new Host, and a new Context for that host, install the WEB-INF folder and your new site is up and running.
     
  • Unlike the Catchall Host approach above, it creates its own Context, so the site will get its own WEB-INF file and its own Web.xml, and it is compartmentalized properly, which is a big plus, especially in production.
     
  • It saves you a lot of messing with the conf files.
     
  • If you want more more “CUSTOMIZED” host files, you can still add them. Tomcat will use a Manually created Host element first, and only fall back to the Valve to create a virtual one if it does not exist. That is a very big plus I think. It means I could use Mod_CFML to get a site up and running quickly, using a Virtual Host element ( in memory) and then you can always update the Server.xml (or an file included in the Server.xml) for Tomcat’s next restart

What are some of the possible negatives of using Mod_CFML?

I’ll prefix this with, after playing with it, quite extensively, I was really impressed with it… so these are negatives, but every method has its negatives.
If the first request to Tomcat, quickly after its startup, is to a Host that does not exist, and the MOD_CFML has to use the Valve to create it, it sometimes spits out a blank page. If you hit a site with a Host element in the Server.xml, it seems to wait then process. The valve must be later in the startup process than the normal Hosts being loaded into memory… so if the Valve is not ready, it just white screens on you, refresh and it will load as expected.

The first request to Tomcat for the Host that needs to be created, takes about 10-12 seconds to complete its work. If you intend to never create manual host elements for your sites, and rely on MOD_CFML to create them every time that Tomcat is restarted, you need to be aware that it takes a significant amount of time for that first request. You might want to consider a script that hits all your sites after a restart, or periodically, or something, to make sure its not a customer hitting it first. Once it is created, it flies… so its just the first request.

If you are thinking, it takes 10-12 seconds per site anyways, using mod_cfml will help the startup time… you are partially right… but normally, the startup time is affected by approximately 5 seconds per site (before the performance improvements coming in my next post) plus 3-4 for the first connection to your site. If the first request is to a non existent Host, and its after the Valves are ready for this type of request, it seems to take the 3-4 seconds for a first connection to the site, plus the 10-12 seconds.

When Tomcat is restarted, all of these In Memory Hosts are lost, and they are only created again when needed. This also means you cannot create special rules, like additional Aliases Virtual Directories/Context etc. One plus with that though, if you don’t have 300 Host elements for sites that are no longer used, or rarely used… so that might be a performance saver too. Although, I’m not sure how Tomcat deals with sleeping sites etc… it might explain the 3-4 second INITIALIZATION it seems to go through on “FIRST” connects, even to existing Host elements. Another note, if you need more than the Host Name and the WebRoot in the Host/Context, you can always manually Add a Host element for future use… which I think is awesome.

Another thing is, if you messed up your Apache Conf, and had the right host name… www.gpickin.com but had the wrong folder, www.gpickin.com/comingsoon… updating the Apache conf and restarting Apache will not FIX the Tomcat Host. It is only set on FIRST CONNECT so you would have to restart Tomcat, otherwise, you would be serving static files from one directory and cfml from another, and trust me, that will not work out so good for you. 
I might be wrong about this, but my testing found this to be true. If anyone knows of another way to force the valve to remove the host and add the new one, please let me know… I couldn’t get it to work.

Why does Gavin not use it currently?

This is a good question. I was always advised to manually create the Host element and contexts, and using the built in AJP with ProxyPassMatch worked, why did I need the overhead of a Perl Module. It has very load overhead apparently, and I guess if I ever get that much traffic for any of my servers, I’ll be doing well enough to have a Server.xml DUDE on staff :)
Also, with the complicated mess of trying to run CF9 CF10 and CF11 with multiple versions of Railo and a Railo Cluster, and now Lucee, I didn’t think it would be flexible enough to handle some requests, while leaving others alone. I assumed I had to leave the Perl config in the body of the httpd.conf file, and not in each VirtualHost, but I found out I could in fact include it on a per VirtualHost to VirtualHost basis. I also found out, that having that Perl conf in my main httpd.conf file, it didn’t affect my multi-engine setup, I just tested it, with CF9, CF10, CF11, and Lucee… all the code all in apache all at the same time… seemed to work just fine.

Is Gavin going to start using it?

Absolutely… most of the reasons I had for not using it have changed, or I misunderstood to begin with. Now I have tested it more thoroughly, I have a much better understanding, of the pros and cons, and this is how I’ll be using it and why.

  1. No restart - I can get up and running with a new site, and if I’m adding it, I’m testing it, so the first load, will be me… I can wait a few seconds.
  2. I can manually create a Host element and Context for all the speed and flexibility long term, after Tomcat restarts, it will kick in.
  3. I can use it with all my crazy engine configurations and it doesn’t seem to have any issues.
  4. Its easier than the relative pathing in my apache conf, and you do not have to worry about all of the quirks of using relatives paths

Thanks to everyone who kept asking me about it. It made me really dive in and see what was “different”. I think I have given it a pretty good test run… and found a lot of pros and cons, of this solution, and the others.

If I missed anything, or anything is not accurate, please let me know.

Next post, will be how to drop your Tomcat startup time from minutes to seconds.

Next problem to solve, how the hell can we get ErrorDocument 404 /404.cfm pass all the information we need to make a useful 404.cfm. Apache usually passes Headers to your 404 file is it is on the same server, and getting this to translate over to Lucee is troublesome. 
We have been playing with this a bit, we’re trying to not introduce another complication into the problem, if you want to chime in, join the conversation here.
Lucee Google Group Discussion - https://groups.google.com/forum/#!topic/lucee/AAwD8wX1k18
 

11
February 2015

Gavin Pickin

How to Work with WebSQL and Managing Multiple Deferred and Promises

Javascript, jQuery, Techie Gotchas, WebSQL

My last post with WebSQL looked at the difference between sync and async, and how you can use Deferred and Promises. The example I showed was a simple query, so we create a deferred, run the query, when the query is completed, it resolves the deferred… and our $.when is waiting for that to complete to move on. That works great in this simple situation, what if you have to loop through 50 items, and have to update each one, and after all of them have done, you want to query the list and display those updates… how do we do that? Its easier than you think. I searched over several blogs and stack overflow questions, API docs, and got it working. I thought you would love a step by step simple example to solve this problem, so here it is.

EDIT: Ray Camden pointed out in the comments, that the Apply function is a Javascript Function, not a jQuery function - thanks for the feedback and keeping me on track.
jQuery is pretty cool, and it offers a Javascript has a function called .apply( this, array) and this is the tool we use to help use with our when. So instead of our $.when( singleDeferred) for our single deferred, we want to $.when ( all 50 of our deferred ). What I have done, is created an array of deferred, and you can call your when like this

$.when.apply( null, deferredArray ).done( function(){
     console.log( 'all inserts are done… reload the list now' );
});

 

Simple but effective… now, how do I load up the array? Let’s look at an example where we have some JSON loaded from the server and we’re inserting into our local WebSQL.

processSync: function( data ){
      var JSONData = JSON.parse( data );     
      var deferredArray = [];               
     for (var i = 0; i < JSONData.data.length; i++){
          this.syncDownTraining( JSONData.data[i], deferredArray );
     }
     
     if ( deferredArray.length > 0 ){
          $.when.apply( null, deferredArray ).done( function(){
               this.reloadList();
          });
     }
     else {
               this.reloadList();
     }
},

syncDownTraining: function( theData, deferredArray ){
      var currentDeferred = this.insertTraining( theData );
      deferredArray.push( currentDeferred );
}

 

I trimmed a little out of it, but essentially, you get some data, and parse it, and loop over it. In each loop, we pass our deferredArray in, when we call the insertData we are returned a deferred, and we add that to our deferredArray.
After we have gone through the loop, we see if the deferredArray length is greater than 0, if so, lets way for the whole array to be resolved, otherwise, lets just process and reload our list.

This is an elegant solution to an interesting problem. 
I hope this helps you, and would have helped past me, and will still help future me.

Thanks for reading.



 

10
February 2015

Gavin Pickin

Guest Blog Posts and Webcasts on CommandBox at Ortus

Chit Chat, CommandBox

With all my recent work and blog posts on CommandBox, the team over at Ortus Solutions, which invested a lot of time and money to bring us CommandBox, invited me to post a few guest blog posts on their Blog to celebrate the 1.0 release of CommandBox. They have several contributors, and will be posting a blog post daily for 4 weeks. Check out my Blog post on their site here - CommandBox - Comments are more useful than you think

On top of that, Fridays at 11am PST, for the next 4 weeks they are having Webcasts, so if you want to watch some videos you can find the previous recordings here https://www.ortussolutions.com/products/commandbox/commandbox-connection
Hint hint - I might even be doing a video, follow me on twitter to find out when @gpickin

 

08
February 2015

Gavin Pickin

Apache and Tomcat - Save yourself the XML Editing - No more Tomcat restarts

Apache, CFML Server, Lucee, Migrating to Railo, Server Admin, Techie Gotchas

On the Lucee Google group the other day, someone mentioned a way to connect Nginx to Tomcat, so you do not need to bother adding the XML Host for every new website you want to run on Lucee (or Railo). This has always been a pain point for switching from Adobe ColdFusion, to Railo or now Lucee, dealing with a Virtual Host for Apache in httpd.conf, and a XML Host in server.xml and making sure they match, and editing requires a restart. Adobe’s ColdFusion runs on a hacked Tomcat, that they specially tuned to work like jrun used to, but a lot of people don’t like being stuck on their upgrade path too. I decided to look at this Nginx approach, and get it running in Apache, and see what downsides I could find.

I played for a couple of hours, and came up with this. I posted my findings to the Lucee group, and haven’t got much feedback on it, so I assume thats a good sign. If you see anything alarming in what I say below, please let me know. I intend to start using this in the dev environment for sure, and maybe even production.

So why should we care about editing 2 conf files, and restarting both engines?

If you have only a few sites, restarting Tomcat is not painful, but if you have 10+, and do no special start up tuning, you can run 30 seconds to several minutes for Tomcat to restart. Note, I have a few tricks that got my restart from 3 minutess to 3 seconds, but that's for another post. Of course this is downtime for your users, whether it is short or long, we want to limit this where possible, and Apache restarts seem quick, and usually not too damaging to the user experience. Regardless, if you are adding a site, you have to restart or reload Apache, so why reload another system too, right?

Conf files aren’t much fun… and having another please to edit settings can get confusing fast, especially if you don’t edit them often.
I am building a CommandBox CLI tool to help with this, which will be looked at more in the next post, but lets see how we can make our lives easier.

FYI - I’m assuming you have a default Lucee setup in this article, so you have AJP on 8009, HTTP on 8888, so if you see these ports numbers, this is why I'm using these.

If you do not want the explanation… skip to the bottom for how to do it. 
I like to UNDERSTAND not just REMEMBER, but the choice is yours.

So how does a normal request happen, lets look at the rough flow, and see how this approach can help.

  • User entered a URL in their browser.
  • DNS translates the URL to an IP and the request is routed to your Web Server
  • Apache handles the request to www.gpickin.com on port 80.
  • Using VirtualHosts, apache looks for a ServerName or ServerAlias matching www.gpickin.com for port 80. It finds one, looking like this

<VirtualHost *:80>
     ServerName www.gpickin.com
     DocumentRoot /www/www.gpickin.com
<Proxy *>
Allow from 127.0.0.1
</Proxy>
ProxyPreserveHost On
ProxyPassMatch ^/(.+\.cf[cm])(/.*)?$ ajp://localhost:8009/$1$2
</VirtualHost>

  • Using this configuration for the VirtualHost, Apache deals with the requests.
  • The Browser might want the favicon.ico that does not match the regex in ProxyPassMatch, so Apache looks for in the DocumentRoot and serves the favicon.ico if available, or a 404 instead.
  • If the default file is index.cfm Apache sees we have a ProxyPassMatch setup, does index.cfm match? Yes.
  • So Apache sends the request via AJP ( Apache Serv Protocol - an efficient trimmed down version of HTTP ) to Tomcat on Localhost looking at AJP port 8009
  • It passes path and query string in $1 and $2 from the regex.
  • When tomcat is done, body content and headers are passed back to Apache, which returns them to the Browser and the user.

This whole process does not really change in the new method, but a slight tweak affects how Tomcat deals with it.

How does Tomcat deal with the request

As we said, listening on localhost port 8009 Tomcat gets a url request. www.gpickin.com/index.cfm

Tomcat looks in its host files, to see how to deal with this. Traditionally, I would have a Host context setup for every site, and it would look like this.

<Host name=“www.gpickin.com" appBase="webapps">
       <Context path="" docBase="/www/www.gpickin.com" />
</Host>

Tomcat would find this Host and match the name, and then would serve the files, treating the docBase as ROOT of this Context/Website/App and process and return the headers/body as needed.

What happens if Tomcat doesn’t match a Host context?

There is a setting for that, on the Engine element, that looks like this
<Engine name="Catalina" defaultHost="localhost">

Usually its set like this, so if it cannot match www.gpickin.com it would use the Host Context for localhost.
There is one by default.. that looks like this.

<Host name="localhost"  appBase="webapps"  unpackWARs="true" autoDeploy="true">
        <!-- SingleSignOn valve, share authentication between web applications
             Documentation at: /docs/config/valve.html -->
        <!--
        <Valve className="org.apache.catalina.authenticator.SingleSignOn" />
        -->
        <!-- Access log processes all example.
             Documentation at: /docs/config/valve.html
             Note: The pattern used is equivalent to using pattern="common" -->
        <Valve className="org.apache.catalina.valves.AccessLogValve" directory="logs"
               prefix="localhost_access_log" suffix=".txt"
               pattern="%h %l %u %t &quot;%r&quot; %s %b" />
      </Host>

So if it didn’t match www.gpickin.com, the Engine element tells it use localhost’s context, which doesn’t have a docBase set, so it uses ROOT inside webApps, and will serve the index.cfm from there. Which would be the Lucee welcome page. Not exactly what we want when we type in www.gpickin.com

The Solution

The solution, using a single context… but how will Tomcat know how to serve the right files? Right now, it will always serve the files out of the Tomcat install dir /webApps/ROOT/
We setup Tomcat’s default context as the root context, with all the websites being sub directories of the root context.
All of my websites, on my dev machine, are in the /www folder. Lets delete the main context and set up the context with a docBase of /www

<Host name="localhost" appBase="webapps">
       <Context path="" docBase="/www/" />
</Host>

Now, we can restart tomcat 1 time, and as we add more sites, we will not touch Server.xml or this host again.
What we’ll do is update the ProxyPassMatch to hit this context and include the relative path, from /www to the Website we want to serve our files.

<VirtualHost *:80>
     ServerName www.gpickin.com
     DocumentRoot /www/www.gpickin.com
<Proxy *>
Allow from 127.0.0.1
</Proxy>
ProxyPreserveHost On
ProxyPassMatch ^/(.+\.cf[cm])(/.*)?$ ajp://localhost:8009/www.gpickin.com/$1$2
</VirtualHost>

See the subtle change?

When we used to call ajp://localhost:8009/$1$2 it would serve from /www
With ajp://localhost:8009/www.gpickin.com/$1$2 it will serve from /www/www.gpickin.com/
If we have a crazy path, we could use ajp://localhost:8009/www.gpickin.com/cfml/wwwroot/$1$2

It doesn’t matter, the context uses the relative path, like one giant website. 
www.gpickin.com/myadmin is translated to ajp://localhost:8009/www.gpickin.com/cfml/wwwroot/myadmin/

So now, Apache serves html out of its root, Tomcat/Lucee serves your cfml out of its root + relative and everything works without add Host Contexts to Lucee/Tomcat, and no more restarts

Wow, that's really simple - but what’s the Catch

Of course, there is always a catch. It didn’t take me too long before I found a couple of gotchas. The fix was easy, so read on.
Component Paths - If I am in the root of my app, and I want to use Component Paths to get to my services.factory… it works. If I am in my models, and I want to, I user services.factory, and it doesn’t work. The reason is component paths search relative first:

  • in /www/www.gpickin.com using services.factory looks in /www/www.gpickin.com/services for factory
  • in /www/www.gpickin.com/models/ using services.factory looks in /www/www.gpickin.com/models/services for factory, then it looks in /www/services for factory.

Of course you could use some DI framework, but the simple fix is set your mapping in the Application.cfc
this.mappings[ "/“ ] = getDirectoryFromPath( getCurrentTemplatePath() );

The other gotcha was in cfincludes, since the Context docBase is /www, if I do a cfinclude template=“/index.cfm” it looks in /www/index.cfm not /www/www.gpickin.com/index.cfm
The mapping works for component paths, as well as cfincludes, so this fix solves both the little issues I found.

Are they the only catches?

You tell me, so far, that's been the gotchas I have found, but I haven’t used it extensively, but if you reset the root in the Application itself, I do not see why it wouldn’t translate from that point on to everything else in your code. If you find anything, or know of anything else, please share, hit me up on twitter, email, or comment right here.
It seems to be a simple but effective fix, eliminating some of the biggest negatives for moving to Railo/Lucee, and if you were already using AJP with ProxyPass, its only a few more characters of code. Of course, you can use the HTTP protocol and ports too, I tested that, and it worked the same. Although, I have heard use AJP, although you might not see the different, in bigger scale its more performant, etc.

Is it a good idea to run everything in one context?

As usual, its one of those answers, it depends. I am not an expert by any stretch of the imagination, and I do not know all the pros and cons of sharing contexts versus each having their own. I believe having separate contexts can be a good thing, although having the choice to group smaller sites, or adding sites quickly using this method has some big wins too. You can always migrate bigger sites to their own contexts, but you can do that at 2am or sometime when it least effects your customers. If someone is an expert that can give us pros and cons, please do, otherwise, do some more research, I know I will.

I think this is a big plus for Dev setups as least, and I know we’ll be using it in production soon, even if its only a temporary measure, so we can add the context and restart tomcat at a time that suits us, but doesn’t slow down our normal procedures and ability to launch a site mid day etc.
Please give me feedback… I hope I covered it all, I hope its accurate, and helpful, and if not, I would like to be able to make it better.

Try it out and come back to find out about my awesome CommandBox CLI tool I’m building, to make this even easier.

Thanks for reading.

05
February 2015

Gavin Pickin

Using CommandBox CLI to build your own Commands for Lucee

Apache, CFML Language, CFML Server, CommandBox, Lucee, Server Admin, Tools and IDEs

Yesterday I apparently started a series of posts, on CommandBox CLI, and how to write your own commands, in cfml, to make your life easier. I started a set of commands under the kiwiSays namespace, and we built 3 commands / cfcs, called addWebsite, startLucee and stopLucee. We got most of addWebsite built, today lets look at startLucee and stopLucee.

I setup a github repo for the project, as we progress, hopefully its useful to see the code… I’ll make a new branch as we create new blog posts. You can find the repo here.
https://github.com/gpickin/cbcli-command-kiwiSays

Before we get too far into the new commands, lets refactor our first one, so its cleaner.

component extends="commandbox.system.BaseCommand" {
        
    /**
     * @websiteURL.hint The Website URL
     * @websitePath.hint Path to the Website Directory
     */
   
    function run( required string websiteURL, required string websitePath ){
             ARGUMENTS.websitePath = fileSystemUtil.resolvePath( ARGUMENTS.websitePath );
    
             var apacheConf = "";
                apacheConf = apacheConf & '<VirtualHost *:80>#CR#';
                apacheConf = apacheConf & 'ServerAdmin myemail@mydomain.com#CR#';
                apacheConf = apacheConf & 'DocumentRoot "#websitePath#"#CR#';
                apacheConf = apacheConf & 'ServerName #websiteURL##CR#';
                apacheConf = apacheConf & 'Include /www/_servers/conf/inc_lucee_conn.inc#CR#';
                apacheConf = apacheConf & '</VirtualHost>#CR#';

                print.line().line( apacheConf );
                fileWrite( '/www/apacheconfs/#websiteURL#.conf', apacheConf );
                runCommand( "run 'sudo apachectl restart’" );
    }
}

 

Everything is jammed in the run function, and while that works, its not really neat and might get harder to maintain, as we build upon this down the road. Lets break it up some, by including other functions in our cfc. Lets pull out the Apache Conf generation, and put it in a new function, called generateApacheConf.

component extends="commandbox.system.BaseCommand" {
        
    /**
     * @websiteURL.hint The Website URL
     * @websitePath.hint Path to the Website Directory
     */
   
    function run( required string websiteURL, required string websitePath ){
     ARGUMENTS.websitePath = fileSystemUtil.resolvePath( ARGUMENTS.websitePath );
    
     generateApacheConf( ARGUMENTS.websiteURL, ARGUMENTS.websitePath);
        runCommand( "run 'sudo apachectl restart’" );
    }
    function generateApacheConf( required string websiteURL, required string websitePath ){
   
    var apacheConf = "";
apacheConf = apacheConf & '<VirtualHost *:80>#CR#';
apacheConf = apacheConf & 'ServerAdmin myemail@mydomain.com#CR#';
apacheConf = apacheConf & 'DocumentRoot "#websitePath#"#CR#';
apacheConf = apacheConf & 'ServerName #websiteURL##CR#';
apacheConf = apacheConf & 'Include /www/_servers/conf/inc_lucee_conn.inc#CR#';
apacheConf = apacheConf & '</VirtualHost>#CR#';

print.line().line( apacheConf );
fileWrite( '/www/apacheconfs/#websiteURL#.conf', apacheConf );
   
    }
}

 

Now, our run function, is more like a controller… calls out to other functions to manage the dirty work. 
That is how we are going the next commands in this super command. We’re going to call the stopLucee command, and the startLucee command from the addWebsite command.

We could right the code in here, but if we want to call them separately too, it just makes sense to have the code in one place, and share that code. DRY Don’t repeat yourself, right.

If you remember from yesterday, when we wanted to run a command, that ran a command in the native shell, we had to call runCommand( run ‘the command’). Today we’re going to call another commandBox command directly, so we only have to use one level of run. So lets add the following code.

runCommand( "kiwiSays stopLucee");
runCommand( "kiwiSays startLucee");

Now, if we restart commandBox, and run kiwiSays addWebsite, it will:

  • generate the Apache Conf from our function inside of our addWebsite command
  • then run a command that runs a native shell command to restart apache
  • then call kiwiSays stopLucee - which is a hello world
  • then call kiwiSays startLucee - which is a hello world too.

Great… that works, and your addWebsite.cfc file should look like this

component extends="commandbox.system.BaseCommand" {
        
    /**
    * @websiteURL.hint The Website URL
    * @websitePath.hint Path to the Website Directory
    */
    function run( required string websiteURL, required string websitePath ){
            
            ARGUMENTS.websitePath = fileSystemUtil.resolvePath( ARGUMENTS.websitePath );

            generateApacheConf( ARGUMENTS.websiteURL, ARGUMENTS.websitePath);

            runCommand( "run 'sudo apachectl restart’" );
            runCommand( "kiwiSays stopLucee");
            runCommand( "kiwiSays startLucee");
    }
   
    function generateApacheConf( required string websiteURL, required string websitePath ){
   
        var apacheConf = "";
        apacheConf = apacheConf & '<VirtualHost *:80>#CR#';
        apacheConf = apacheConf & 'ServerAdmin myemail@mydomain.com#CR#';
        apacheConf = apacheConf & 'DocumentRoot "#websitePath#"#CR#';
        apacheConf = apacheConf & 'ServerName #websiteURL##CR#';
        apacheConf = apacheConf & 'Include /www/_servers/conf/inc_lucee_conn.inc#CR#';
        apacheConf = apacheConf & '</VirtualHost>#CR#';

        print.line().line( apacheConf );
        fileWrite( '/www/apacheconfs/#websiteURL#.conf', apacheConf );
   
    }
}

 

Lets get our stopLucee and startLucee commands fleshed out. As I said yesterday, the first iteration of these commands will use hardcoded paths etc, which of course, you can change to suit your needs. Later, we will look at other ways of making it more flexible, useable, and distributable… especially across platform. For now, play along.
stopLucee is a simple command. We’re going to use a variable for the location of the path of our lucee install. We’ll print a line to show progress, and identify what we’re doing, i.e. stopping lucee running from this install path, and then, we’ll run the command.

component extends="commandbox.system.BaseCommand" {

     function run() {
          var serverPath = '/www/_servers/lucee';
          print.line('Stopping Lucee - ' & serverPath);
          runCommand( "run '#serverPath#/bin/shutdown.sh'" );
     }
}

 

Not much to it, but thats all we need. 
Lets copy paste and update for a startLucee command.

component extends="commandbox.system.BaseCommand" {

     function run() {
          var serverPath = '/www/_servers/lucee';
          print.line('Starting Lucee - ' & serverPath);
          runCommand( "run '#serverPath#/bin/startup.sh'" );
     }
}

 

Lets restart CommandBox and try it out.

$ kiwiSays addWebsite
Enter websiteURL (The Website URL) :www.gpickin.com
Enter websitePath (Path to the Website Directory) :/www/www.gpickin.com
<VirtualHost *:80>
ServerAdmin myemail@mydomain.com
DocumentRoot "/www/www.gpickin.com"
ServerName www.gpickin.com
Include /www/_servers/conf/inc_lucee_conn.inc
</VirtualHost>
Command completed succesfully!
Stopping Lucee - /www/_servers/lucee
Command completed succesfully!
Starting Lucee - /www/_servers/lucee
Command completed succesfully!

Awesome… now we have a single command, that calls other commands, creates conf files, restarts apache, stops and then starts Lucee, pretty easy.

You might be asking, why are we stopping and restarting Lucee… we haven’t made any changes to server.xml… or my special inc_lucee_hosts.xml file I mentioned in my post titled "Adding Goodies to Server.xml to make life easier with Lucee” http://www.gpickin.com/index.cfm/blog/adding-goodies-to-serverxml-to-make-life-easier-with-lucee well, we have to leave something for the next post, right?

So thanks for reading, hope you check back in for the next post in the series.
Before Files - https://github.com/gpickin/cbcli-command-kiwiSays/tree/Blog-Post-1
After Files - https://github.com/gpickin/cbcli-command-kiwiSays/tree/Blog-Post-2
 

Blog Search