Blog

Blog Archives - 9 Record(s)

Year: '2014' - Month: '3'

22
March 2014

Gavin Pickin

Conferences - Amazon AWS Summit in San Francisco and Brackets Meetup at Adobe

Chit Chat, Conferences, Server Admin, Steals and Deals, Tools and IDEs

Its a busy week next week, driving to LAX and back Monday with my Family flying in to spend some time with the kids, Tuesday, I'm driving to San Francisco for an action packed Wednesday. If you didn't read the title, then I'll spell it out to you.. Wednesday, 26th of March 2014, Amazon has their Free Amazon AWS Summit. Its a day packed full of amazing sessions, so I can learn anything and everything I need to about everything AWS... and after that, I have to decide between Amazon Social and Brackets first Hackathon.

Amazon is having these AWS Summit's around the world, and around the country, and they have Bootcamps and Certification Exams the day prior. There are social gatherings for Startups, Women in Technology, and a general Social as well. There are hands on sessions, breakout sessions for more details, and Partner Solutions to look through as well. Its going to be a day of tough decisions, but going was an easy one to make.

For more information, read this, there are still spots available, if you are in the area, I think its well worth a look.
https://aws.amazon.com/aws-summit-2014/san-francisco/

After a day full of Amazon AWS goodness, there is a social, which sounds appealing, but there is something else that showed up on Twitter just the other day. Brackets and the Adobe Brackets team is having a meet up / hackathon, at the Adobe Headquarters right afterwards. 

I've stolen a little text from the meet up, here it is

SF Adobe office, 601 Townsend St 

Meet the Team and hack on Brackets.

Join us for a mix of talks, networking, and good company at our first Brackets Hackathon. We'll also have pizza and beer and some coveted Brackets swag to give out!

The Brackets team will give you info on: 

• What is Brackets and how can it help you code the web

• How to hack Brackets and create the next great Brackets Extension

Adobe's Narciso Jaramillo (NJ), Peter Flynn, Kevin Dangoor and Jonathan Dunlap will talk about Brackets, its future and how to get started hacking on Brackets. 

http://www.meetup.com/Brackets-Editor/events/169625592/

Its going to be a busy Wednesday, if you are around, or in driving distance, join in the fun.
If you aren't around, I guess you'll have to wait for the report afterwards.

Hope to see you there,

Gavin

21
March 2014

Gavin Pickin

Conferences - 2014 NC DevCon - Have you say and make it the best NC DevCon yet

CFML Language, Chit Chat, Conferences

Conference season is about to go into full swing, especially for the ColdFusion community. cf.Objective() and Into The Box are coming up quickly in May, get your tickets and book your flights. Scotch on the Rocks is down to 14 tickets last I saw, so if you intend to go, snatch one up quick. Next in the line of CF Conferences I believe is NC DevCon, and they have released a survey, to get the community input, to make this years NC DevCon as good as can be.

They're asking for ideas on track ideas, and more specifics on given tracks to what type of content for those categories. They also want to know if there are certain sponsors you want to see there or not, and most importantly, whats days make it more appealing.

Take your time, and fill out the survey, the more information, the more the committee can do to shape it to meet our wants and needs.Its a short survey, well worth the time, so fill it out, and spread the word.

https://docs.google.com/spreadsheet/viewform?formkey=dFdVcWhBb2xJWTVSZFIybHJxWEh1Unc6MA

19
March 2014

Gavin Pickin

Cold Fusion Builder - A seriously destructive bug makes it into Version 3

CFML Language, CFML Server, Techie Gotchas, Tools and IDEs

I mentioned in a previous blog post about some Bugs in CF Builder, although, overall, so much happier with the product than I was with previous versions. The quick file closing is amazing, something I would always dread, but that is gone now, so thats a big plus. I am also seeing less drag on the intellisense like features and lookups, so speed is improving on the whole, but some lil editor bugs here and there.

The big items that I hated in 2, is still in 3... and has caused some headache, heartache, and seriously pissed off several devs I know. I updated the bug today, since Adobe couldn't reproduce it, and I put a video on Youtube to explain the process and show the bug itself, at its destructive best.

Sure, its not going to affect many people, you have to be using ColdFusion Builder, and using RDS. RDS is frowned upon in Production of course, but on dev and staging it is useful to activate for a short time at times, even if only locally. I urge you to go vote for it, watch the video, and you'll see how its a pretty nasty bug, and should be fixed. Those without source control / backups could seriously hurt themselves, think about accidentally deleting your whole entire folder structure... ouch.

Bug:
https://bugbase.adobe.com/index.cfm?event=bug&id=3722406

Youtube Video:
https://www.youtube.com/watch?v=-r5Z_XcbHf4

13
March 2014

Gavin Pickin

CFML Language - Talking to Elishia Dvorak from Adobe + ColdFusion about the Edu + Intern Program

CFML Language, CFML Server, Online Interactive Learning

Recently on Twitter, there was some talk about today's ColdFusion evangelist, and who that is. Eventually we tracked down Elishia Dvorak @elishdvorak, who is Adobe and ColdFusion's US representative, who deals with Sales, and the new Education Program, as well as support and everything else. Once we had identified her, we gave her a few jabs about her number of tweets, and then let her be. Elishia posted a tweet

EDU initiative. Interested companies willing to offer internships to new #ColdFusion developers, DM me.

Of course I responded and said "sure, we're interested". I always like to learn what is available and what the company driving our community is doing to help promote the language, and of course bring fresh blood in. We exchanged emails and the following day, she called me, and we had a pretty good talk, so I wanted to share the discussion.

Note: I wasn't told any of this is secret, or privileged information, so I assume its free to discuss, and hopefully discussing and promoting what Adobe is doing will get more people involved.

First of all, we discussed the Edu Initiative. The ColdFusion team has worked hard to create a whole Web Development curriculum. This involves front end and back end technology, with the backend technology using ColdFusion of course. The course is designed as an introductory course to start with, and by the time the student is finished with the 1 semester course, they are familiar with the language, some of the work flow, and how things work together. 

This Initiative needs your help, and you will be rewarded. Adobe is asking for community members to help find universities and colleges (community colleges seem to be the best fit) that this class might work well in, and if you can provide them with contact details for someone to discuss this, basically, create a lead, if its a good lead, or the lead converts, Adobe will reward you with an iPad, and according to his Instagram photo, his was engraved with ColdFusion Rockstar

http://instagram.com/p/lJWvCGO_1y/

So, if you know anyone at a local institute, or know of a program that might be a good fit, pass on whatever info you have to Elishia and she can follow up for you.

By itself, I think its a good idea, it would be nice to have more content obviously, but its a great start. Then we got into discussing conferences etc, and I would say hi at cf.Objective(), as I am going, of course I'm presenting. She sounded interested at that point, and we started to discuss the next step in their plan.

I started ColdFusion way back in Version 4, and in the old days, they had a Fasttrack to ColdFusion and a Bootcamp style advanced Coldfusion course. These were around for some time, but I think the trainings pretty much died off, and the Wack Books took over. I used the copy I had of this course and trained several CFers over the years, and was thinking it would be nice to have training materials to help mentor newbies in, much like the Learn CF in a Week. Elishia mentioned that they're going to spend some time and try to revisit these courses, and try and create their own, new and improved ColdFusion BOOTCAMP. This isn't finalized yet, its more of an idea, and I'm not promising anything, but I think its a good idea, and showing support of the idea might help make sure they implement it. The idea was, they would create a curriculum of sorts, that they could give to instructors (yes, community members like ourselves) to run Adobe BootCamps on their behalf. So with their assistance, we could run 2 day bootcamps to anyone willing to learn, and they mentioned they'd like to make the barrier to entry as low as possible, hopefully free. This would really help drive some fresh blood into the community.

I was impressed with the idea, of course, its not ready to implement yet, but its a solid idea, and I think we should tell them that, so they know it, and know they have the support.

Finally, after that discussion, we finally got to the point of the talk, the INTERNSHIP talk. 

To make the EDU initiative successful, and appealing to these Institutions, they want to know that if their students complete these courses, that they have jobs waiting for them. Adobe is wanting to build up a list of companies that would be willing to take Interns in some type of Intern program, so the colleges can help feed you their best students, and gives the course more appeal to the institute, and those students taking the course. Adobe isn't planning on regulating the Internship program, so it sounds like its really up to your company how you implement it, if its just a summer program, etc. The good news is, with the Bootcamp, you could use the same curriculum to help get your Intern going, and they would be coming from an Institute with this program, where they would feed you the better students.

As well as giving Elishia (@elishdvorak) my approval for the Internship program, I also gave her a few more contacts for the local Community College, and the local Cal State for her to follow up on. My previous employer, who uses a LOT of student employees would also be interested in the Intern Program, so I passed that information on in good spirits too. More CFML Developers is good for everyone.

We also discussed the Betas, and the Bug Tracker and promised to chase down some bugs internally for me, especially the large RDS bug I reported again the other day, will post about that one in the next few days too. It was a great talk, enjoyed it, and impressed with where they are, and where they intend to go. So lets all chip in and do what we can.

Take a step out and help out the community, there is a lot of benefits for you, your company, and the community at large, so reach out and talk to Elishia (@elishdvorak). Tell her Gavin sent you :) Then I can add another restraining order to the CF Hour one :) 

Have a good one,
Gavin

11
March 2014

Gavin Pickin

Mobile Development - JSON vs JSONP with Ajax calls and ColdFusion CFC Components

Android, CFML Language, Cordova / Phonegap, IOS, jQuery, Techie Gotchas

I had mentioned that I am getting setup and writing some HTML5 / Javascript / CSS / Phonegap / Cordova Apps, and along with these types of apps, you will start to use Ajax to make server api calls to keep your fresh and up to date. I ran into a hurdle today when making some Cross Site Ajax calls, and decided I would share my findings.

First, you might ask, if I'm making a native app, why are you worrying about Cross Site Ajax calls? And you would be correct, with the app deployed on a Android or IOS device, those Cross Site Scripting issues do not exist. I decided I wanted to develop my apps to be deployed on our dev servers during development and testing, and wanted them to work as well as they could (obviously certain device api's would not be available), so I wanted to look into using JSONP to solve my cross site scripting issues. Using json a lot, and with most of my sites calling the same server for api calls, I have heard of JSONP, but not actually implemented it myself… so here goes.

Why JSONP, or maybe What is JSONP?

EDIT: Ray Camden just reposted an old article that had been removed, that might explain it better than me. Thanks Ray for posting it again. 
http://www.raymondcamden.com/index.cfm/2014/3/12/Reprint-What-in-the-heck-is-JSONP-and-why-would-you-use-it

JSONP is basically a workaround that bypasses cross domain scripting policies. Instead of making an ajax call and requesting json, you actually create a script tag, with the get call as the SRC attributes of the script tag, which is allowed to call across domains, the difference is, instead of JSON, you get json wrapped in a function call, so when the script tag completes, it runs the call which basically is passing the json you wanted in the first place. Instead of returning an object that looks like this

{ "Name": "Gavin", "Height": "6'5" }

 

You would get something like this

callBackFunction({ "Name": "Gavin", "Height": "6'5" });

 

Sounds complicated? It is, but guess what, jQuery makes it easy… because all you do is change the dataType from json, to jsonp and jQuery behind the scenes creates the script tag, creates a callback function, appends the name of the callback to your ajax url, and all you have to do it make sure the server you are hitting returns jsonp, and not json.

What does the javascript look like for a same site json request

$.ajax({
    url : "http://servername.com/cfcs/User.cfc?method=getUser&user="+userid,
    type : "GET",
    dataType: 'json'})
   
    .done( function(data){
         alert('success');
     });

What does the javascript look like for a cross site jsonp request

$.ajax({
    url : "http://servername.com/cfcs/User.cfc?method=getUser&user="+userid,
    type : "GET",
    dataType: 'jsonp'})
   
    .done( function(data){
          alert('success');
    });

  

Yup, thats it, just need to change the json to jsonp… and jQuery does the rest.

 

I setup my cfc to get the call, and return my json, and I got a strange error. 
In Chrome, the message was this if I passed back returnformat="plain" and returntype="string"

Resource interpreted as Script but transferred with MIME type text/plain: "http://servername.com/cfcs/User.cfc?method=getUser&userid=1234&callback=jQuery110207742651014123112_1394599853046&_=1394599853047". jquery-1.10.2.js:8516

Uncaught SyntaxError: Unexpected token :

In Firefox, the message was this

SyntaxError: missing ; before statement

 

Now, if i set returnformat="json" and returntype="string" , the ajax call still fails, but the console has no error messages at all from Chrome or Firefox.
If I added a .fail into the ajax call, I can see there is a parse error, but it was not the easiest thing to debug. This is one of the big issues with jsonp… because you're abstracting the actual call behind this callback, the debugging is not as easy to follow, and just isn't as helpful as it might be in a normal json call. 

I thought maybe I had invalid JSON, maybe I had invalid characters like Ben Nadel had mentioned in a post. 

I knew that jQuery did all the work, but what I didn't realize was my CFC wasn't passing back a jsonp response, and although its obvious now, at first, debugging it was really pretty hard. The fact I didn't know that the jsonp response had to be constructed differently was my mistake, I guess I assumed jQuery was so awesome, it could fix my server side code too. As you can tell, from above, the error messages weren't obvious… so that is why I wanted to share my experience.

I modified my cfc function from this

<cffunction name="getUser" access="remote" returntype="string" returnformat="plain" output="false">
    <cfscript>
        if ( structKeyExists(url, "agentid")) {
            return '{"Code": "200", "ID": "12345"}';
        }
        else {
            return '{"Code":"400","ID":"None"}';
        }
    </cfscript>
</cffunction>

 

to this

<cffunction name="getUser" access="remote" returntype="string" returnformat="plain" output="false">
    <cfscript>
        if ( structKeyExists(url, "agentid")) {
            return '#arguments.callback#({"Code": "200", "ID": "12345"});';
        }
        else {
            return '#arguments.callback#({"Code":"400","ID":"None"});';
        }
    </cfscript>
</cffunction>

 

And it works.
You can see the in one of my error messages, you see the full url.

http://servername.com/cfcs/User.cfc?method=getUser&userid=1234&callback=jQuery110207742651014123112_1394599853046&_=1394599853047

jQuery had created this callback function called jQuery110207742651014123112_1394599853046 and added that to the url as the value to the callback key pair. To respond with jsonp instead of json, I simply needed to respond with a valid line of javascript, to be executed like any other line of javascript, which calls a method, by the name of the value of the callback passing in 1 variable, the json response, and ending it with a semi colon for good measure.
So my page should return

jQuery110207742651014123112_1394599853046( {"Code": "200", "ID": "12345"} );

When this returns, the page executes this function… which passes the data to my success function, and nothing else on the client side had to change.

Remember, this is not json, its a line of code, so your function should return format="plain" not json. This obviously means that you cannot use ColdFusion's built in serialization by using the returnformat="json" when you need to return jsonp, so you might want to call a second function for jsonp calls, that just wraps your normal function with a jsonp function.

<cffunction name="getUserP" access="remote" returntype="string" returnformat="plain" output="false">
    <cfreturn '#arguments.callback#( #getUser(argumentCollection=arguments)# );'>
</cffunction>

 

Then when you want json, call getUser, and when you want jsonp call getUserP. That way the mime types match, and you can make use of the normal return format="json" for your normal method, and you're keeping to the DRY Don't Repeat Yourself rule at the same time.

 

Hope this helps someone looking at JSONP for the first time, thanks for reading,

Gavin

10
March 2014

Gavin Pickin

Charity Corner - Diabetes and Fly a Foul Mouthed Fusioner Fund

CFML Language, cfObjective, Charity Corner, Chit Chat

EDIT - Adam Cameron has decided there is a GREATER  need than his own.
Jared, the creator of cf.Objective() has a gofundme page, here.
Adam has instructed me to donate the funds acquired so far and donate them to Jared, and if you would like your donation back, please notify Adam, and he will refund the donation back to you.
Please help a fellow techie in need, spread the word.

Tomorrow I have a meeting with one of my new Charity Corner businesses tomorrow. I have decided to try and help out the community by dedicated so many hours a month to help local charity and non profit groups trying to get up and running. Tomorrow, I'm meeting with a brand new Non Profit who's mission is to help run camps and events for young and old Diabetics (type 1 and 2), in our region, where there are no Diabetic groups currently running. With parents suffering from Type 2, and one of my Sisters a life long Type 1 sufferer, I thought it was a worthy candidate for this months project. While thinking about this Charity project, I thought of another one, that the ColdFusion community might be interested in.

In our small but powerful ColdFusion community, our currency seems to be based on Virtual Thank you's, and Beer… surprisingly, a lot of beer. I am saddened to say, one of our biggest contributors to the community of late was thinking about cashing in some of this currency, but was sad to know, that the Airlines, will not take a series of IOUs, even those IOUs for beer.

Gavin, what the heck are you talking about?

On Twitter last week, our Fowl Mouthed Friend (I say in Jest), Adam Cameron was enquiring abouthotels around the Mall of America, and trying to see if he could stretch the budget to venture out of Bloomington in May, for the best CFML Conference, cf.Objective(). Ray Camden responded pretty quickly to let Adam know he could bunk with him, and several others gave him lots of advice and input, and mentioned how it would be awesome to meet him, or hang out with him.

So that got me thinking… being an avid reader of Adam's blog, and his war stories of late, there are lots of people that owe him a beer, or two, for helping them out with a problem, an issue, troubleshooting, double checking, or just trail blazing the CFML path with CF 11 as he spent his whole vacation punching out a million interesting and thorough blog posts. I think most who read his blog probably owe him a drink or two, and the community as a whole has benefited greatly from having him stand up and be counted in all of the roles he has lately.

  • A thorough, technical, loud but quality blog.
  • Co Created - CF UI The Right Way
  • He Created the Twitter Bug Tracker Scraper Reporter for Updates and Un-Triaged Counts
  • He updated the Bug Tracker to post CFML Bloggers posts to Twitter
  • He is the Admin for CFLib.org site now (created by Ray)
  • He is one of the most prolific Bug Contributors to the Adobe Bug Tracker System
  • He is one of the big contributors to the Adobe Wiki Documentation since the conversion
  • He is the Community Bulldog, and Twitter tyrant, fighting for the CFML Community.

If everyone who thought he had done them a solid, done the community a solid, cashed in a couple of those BEER credits for him… I bet we could Fund a Flight for that Foul Mouthed Fusioner.

I'm sending him a $20, maybe you can too, or maybe someone you know could, heck, the Adobe Team should throw in a ticket to cf.Objective() for all the work he does for them.

Am I Crazy? Yup!
I thought I'd throw it out there though, because if you are not asked, you won't give… and Adam isn't the type to ask.
Here's a paypal account if you want to send a few bucks his way. 

EDIT - Please go to support Jared here instead

Back to Diabetes

As for my other Charity case… I meet with the director tomorrow, and we're going to sponsor the organization's website. We will help them get themselves setup with social media, and give them a push in the right direction. I am a firm believer in the more you give, the more you receive, and organizations like this one need all the help they can get to start with. It never hurts your reputation to be a contributor to the community and a worthy cause either.

How are you helping your community?

Thanks for reading,
Gavin

07
March 2014

Gavin Pickin

Dev Ops - From Mail Logs to DB Stats in a CFML Dashboard - Part 5

CFML Language, Server Admin

Its been a month since my last post in this series, so if you weren't reading my blog then, jump back into late Jan early Feb, and catch up on where we are at. We've done some testing, and we ready to clean up our log file import into the database.

Again, the code is raw, single page, no framework, not the prettiest thing around, but I think I have it pretty well nailed down now.
I have put the file in a gist, and will walk through it, discussing sections, and why I did what.

https://gist.github.com/gpickin/9406303


<cfsetting requesttimeout="600">
<cfparam name="thelogname" default="clean_maillogH.log">
<cfinclude template="stopwatch.cfm">
<cfset ds = "dev_bktools">
<cfset stopwatch = makeStopwatch()>
 
<cfoutput>
 
<cfset stopwatch.start("Begin timing")>

 

Lines 1 - 9.
As I mentioned last time, sometimes I will pass the log name in, because I have 2 logs I am processing, one updated hourly by our cron scripts, and the other, at the end of the week, in which we just check to make sure we don't miss any log items. I initiate our stopwatch, and begin timing.


<cfquery name="ind_control" datasource="#ds#">
	select * from ind_control
	where filename = <cfqueryparam value="#thelogname#" cfsqltype="cf_sql_varchar">
</cfquery>

 

Line 11 - 14
I am querying my control table, based on the name of the file, we want to see if the log we're processing is the same file we last processed, and if it is, where did we get to last time, so we can continue processing.


<cfset thecounter = 1>

 

Line 16 - Simple counter, so we know what line of the log file we're on.


<cfloop file="/pathtomylog/#thelogname#" index="thelog" >

 

Line 17 - We're going to loop through the log, line by line… as we progress, we'll count each line, and use that to compare to our last processed line, and see if we're at the end of the file, or not.


<cfif thecounter eq 1>

 

Line 18 - If we're on the first line of the file, we want to check and see if we have processed this before or not.


<cfset thedate = parseDateFromLine(thelog)>

 

Line 19 - Calls a UDF to take the line as input, and parse out the date/time, and return a date object.


<cfset batchsize = 50000>

 

Line 21 - We set the batch size to 50,000. It seemed to be quite a good size for reading and inserting into the database, it it should only take a few runs to roll through even large logs.


<cfif ind_control.dte_good neq thedate>
	<cfset startrow = 1>

 

Line 22 - 23 - If the date does not match the start date we had in the control file, this is a new file, so we need to start processing from the beginner no matter what the control file says.


<cfelse>
	<cfset startrow = ind_control.int_good_end + 1>
	<cfif ind_control.int_endoffile is 1>
		<cfset endoffile_found = 1>
	</cfif>	
</cfif>	

 

Line 24 - 29 - We set the starting row as the last row we processed, plus 1, and if last time we were at the end of the file, we set a variable to state that, later, we'll check that, to see if the number of rows have changes, and reset it or not.


<cfset endrow = startrow + batchsize - 1>

 

Line 30 - We just set the endow, for this batch run to be the start row + the batch size - 1, so we process the batch size number of records… since the start row is 1, 1 + 50000 is 50001, take one off, to keep nice round numbers.


Now we're setup the start of the file, we can process the actual logs.


<cfif thecounter gte startrow and thecounter lte endrow and not structkeyexists(variables, "endoffile_found")>

 

Line 33 - If we are supposed to process this row, we'll jump in and do some work… if not we skip all the following processing, and just count the line. 

  • We check for if the counter (row number) is greater than equal to the start row. We don't want to process any we have already processed.
  • We check for see if the counter (row number) is less than or equal to our endow, we only want to process those in our batch size
  • We only want to process if there is not an EOF flag. If there is an EOF flag, we'll just count the rows, and at the end, we'll reset the EOF flag in the control table if we found more rows than we thought the log file had (this is mainly for the hourly log, that grows every hour). 

If it is a row we should process, we jump into the next block of code.


<cfset thedate = parseDateFromLine(thelog)>

 

Line 34 - Get the Date/Time so we have it.


<cfquery name="logCheck" datasource="#ds#">
	select id from ind_log
	where dte_log = #thedate_log#
	and txt_log = <cfqueryparam value="#theline#" cfsqltype="cf_sql_longvarchar" >	
</cfquery>

 

Line 36 - 39 Look in the DB for an entry with the same date time stamp, and the actual log line as well. I know just using the log itself would work, but a date time index is faster, than the text index, so by using that as the first where clause, we can speed the look up dramatically.


<cfif logCheck.recordcount eq 0>
	<cfquery datasource="#ds#">
		insert into ind_log
		set 
		dte_log = #thedate_log#,
		txt_log = <cfqueryparam value="#theline#" cfsqltype="cf_sql_longvarchar" >	
	</cfquery>

 

Line 40 - 47 If that record was not found in our DB then we'll insert it into the db, inserting the date time stamp, and the line itself, for future processing. Note, in the hourly processing, with the code setup this way, it should never find a duplicate… but, at the end of the week, the hourly log gets saved as the weeks log, and we will process through that, checking for any that happened to get missed, so we double check. 


<cfelse>
	<p>Duplicate found</p>		
</cfif>

 

Line 48 - 50 state if a duplicate was found.


<cfset thecounter = thecounter + 1>

 

Line 53 - Increments the counter, whether we tried to process it or not. and Line 54 closes out the loop.


</cfloop>
#thecounter#	
<cfset stopwatch.lap("Loop and Save Data")>

 

Line 55 - 57 - We dump the total count of the rows after looping through them all. Not needed for the scheduled task, but if running manually, its useful. We set the lap on the watch.


<cfif endrow gt thecounter>
	<cfset endrow = thecounter>
	<cfset endoffile = 1>
<cfelse>
	<cfset endoffile = 0>	
</cfif>	

 

Line 59 - 64 If the endow was greater than the counter, we processed all the rows in the file, so we mark the end of the file, and set the endow to the number of rows, so if we start processing again, we start where we ended.
If the endow was not greater, we mark the end of file was not reached, and next time this is run, it will continue processing.


<cfif structkeyexists(variables, "endoffile_found")>
	<cfif thecounter gt ind_control.int_good_end>
		<cfset endoffile = 0>
		<cfset endrow = ind_control.int_good_end>
	</cfif>
</cfif>	

 

Line 67 - 72 We check to see if the counter was gt that the last processed row in the control file… if it is, we reset the endoffile, and make sure the endow was the control. We could probably remove this logic, but its initially how I wrote it, although, we could adjust it in the Line 33 block looking back at it.


<cfquery name="ind_control" datasource="#ds#">
	update ind_control
	set dte_good = #thedate#,
	int_good_end = #endrow#,
	int_endoffile = #endoffile#
	where filename = <cfqueryparam value="#thelogname#" cfsqltype="cf_sql_varchar">
</cfquery>

 

Line 74 - 80 - Update the control file… so we know where we got to, and where we're going.


<cfset stopwatch.lap("Saved to Control File")>	

<cfdump var="#stopWatch.getTimeline()#">

 

Line 82 - Lap the clock
Line 84 - Dump the stopwatch output, so we can see what time was spent processing.


<cffunction name="parseDateFromLine">
	<cfargument name="thelog" required="yes">
	
	<cfif left(arguments.thelog,7) eq "maillog">
			<cfset theline = right(arguments.thelog, ( len(arguments.thelog)-find(":", arguments.thelog) ) )>
		<cfelse>
			<cfset theline = arguments.thelog>
		</cfif>		
		<cfif find(' independence ', theline) gt 0>
			<cfset cleanedup_val = left(theline, find(' independence ', theline)-1)>
		<cfelse>
			<cfset cleanedup_val = left(theline, find(' new-independence ', theline)-1)>
		</cfif>			
		<cfset thedate = parsedatetime(dateformat(now(), "yyyy") & " " & cleanedup_val)>
		<cfif thedate gt now()>
			<cfset thedate = dateadd("yyyy", -1, thedate)>
		</cfif>	
		<cfreturn thedate>
</cffunction>		

 

Line 87 - 105 - The UDF Function that takes the line, and parses out the Date.

Our logs have several names, due to the migration. So we look for the start of the line, if it says maillog, we remove that from the start of the line.
We look for the server name, independence, if its there (without the new- in front of it) we grab everything in front of it.
If not, we grab everything in front of new-independence.
We add the year, since the year is not given in the logs. We guess and add this year
We parse the date… if the date is in the future, then the year we added was off by 1 year, so we remove a year from the date.
And then we return the date.


Its a lot of simple tasks, but it seems to be working pretty well.
As I stated, its not the most elegant code (yeah, its in tags too), but its processing the logs, and prepping them for more crunching. 
The schedule task runs every few minutes, after the files have been transferred by the cron tasks. So every hour, new data is fed into the DB.

Next we'll write a couple of shorter scripts that take these log lines, and pull them apart, and save more refined data into new tables. 
I will add a flag to each log row, so we can tell which log rows have been processed, for each type of mined data.

We're almost ready to make that pretty dashboard. 
We're into ColdFusions world now, Code and DBs.

Check back soon for that.
Gavin

05
March 2014

Gavin Pickin

Migrating to Railo - Is ColdFusion Forgiving or Sloppy?

CFML Language, CFML Server, Migrating to Railo, Techie Gotchas

Over the long never ending migrating from ColdFusion to Railo (a lot of legacy sites mentioned in several other blog posts), I have seen a pretty interesting trend. I was discussing with a friend today, who migrated one of their sites from ColdFusion to Railo just this week, and we were discussing the issues they faced, and how they compared to what I have seen… it was interesting, and it got me thinking… run for cover, me thinking is dangerous.

The big trend we found was, most of the migration bugs and issues we faced were not necessarily ColdFusion vs Railo compatibility bugs, but actually the fact that ColdFusion let our programmers, some of the time, me and my friend, code a bunch of garbage, and let us get away with it. You might think, well, thats nice to have a forgiving engine / language, but the more I thought about it, I thought, no, I would rather have it right than forgiving.

The perfect example goes back to my experiences with PHP. When you screw up something in PHP code, unless you change settings, you can run a page, no errors show up on the page, and you think your code is good, and it ran. Then, you realize, looking in some logs, that PHP just plain ignored those errors, and kept processing as best as it could. It drove me nuts, and being new with PHP, I didn't know how to set it to strict mode, so it was frustrating to develop with. Of course, I quickly learned, but the point is, if there is an error in my code, I want you to say "HEY GAVIN, YOU SCREWED UP". Right?

Thats the thing, Railo is pretty strict, and that is out of the box, which, to be honest is actually relaxed, to try and be compatible. Railo threw a bunch of errors with several sites I moved, and under closer investigation, Railo was right in calling the errors, but my thought was, how the hell did ColdFusion not throw an error?

There are several types of errors, so lets look at them, and think about what we expect and shouldn't expect.

First - additional non supported attributes.

This one, I give ColdFusion benefit of the doubt… and I actually think its ok. If you annotate a tag or component, ColdFusion should not throw an error if that attributes is not a valid attribute for the tag. One could argue that if you get the attribute name wrong, it should tell you, but I think annotations and the meta data we can inject into our cfcs etc is useful, so having strict checking on those are not necessary.

Now, I wonder if Railo has a setting that I missed, but I noticed that some of the tags would throw an error if you had unsupported attributes. One of the main ones, was cfinput… most of the ajax attributes like 

typeahead="yes"
showautosuggestloadingicon="yes"
visible="yes"

 

Then I realized that the custom tag replacement cfcs, seemed to be a lot more strict, so I wondered if those CFCs had a much stricter threshold than normal tags etc. When I created a replacement for the cftextarea in rich text, I had to add arguments for width, height, toolbar, class, style, anything and everything that the legacy code might have given to it, otherwise the call would error out. Of course, this is legacy code, and I am thrilled to be able to override the tags at all (great way to replace ColdFusion UI with UI done the right way), and maybe I just don't know the setting to relax those settings, but it is nice to know that a attributes isn't being used, but, at the same time, with annotations etc, its nice to have that flexibility.

So ColdFusion is flexible here, Railo isn't as much (that I am aware), but there is probably a setting, Railo has a lot of flexibility for compatibility and performance optimization, and its probably only for Replacement CFCs.

Second - invalid attributes values

This is one that just plain frustrated me. I see the value of annotations above, but I don't know why it is acceptable for ColdFusion to allow misspelled values for certain attributes of tags and functions. My friend, who shall remain nameless, was telling me how he happened to misspell the CFQueryParam cfsqltype, and ColdFusion just ignored it. To me, there should be a set of values that it can accept. I know Adam Cameron has discussed how Railo simplified the values you could use, and ColdFusion was following suite, but surely, if I put cf_sql_inteRger with an R in the middle of it, that it should throw an error and say, thats not a valid attribute.

ColdFusion doesn't do anything, and the scary part is, since its not valid, does it actually do anything with it? Does it just escape it and treat it like a string, does it guess interger means integer (i doubt it)… I don't know, but it doesn't do what I intended, and it didn't give me any reason to fix it.

Another example of mysterious attribute values, for CFINPUT, I know, another UI Tag… the Validate attributes. You can put telephone, email, several different options, but if you enter

validate="integer,required" you would think that would be invalid, and again, throw an error. So not only was it not requiring the field, it probably wasn't validating the type to be integer, so there are 2 pieces of functionality that were coded incorrectly by the developer, that has stood for years, functioning outside the scope of their intentions, and is invalid, and they never knew. 

Again, correct me if I'm wrong, but I didn't think individual developers could extend this set of values, so if its set, why doesn't it validate them? 

Railo catches this, and now I know there was a bug in my code, and I can fix it. The question is, if that can only take certain values, I don't see any use cases for custom values, unless you're Railo, but of course they could add those to the list, but since they're known, surely they should be checked at compilation. Seems sloppy, not forgiving.

Talk about sloppy, here is some sloppy coding.

You have a query names qCustomers
You have fields, fName, lName email.

<cfoutput>#qCustomers..fname#</cfoutput>

 

Run this in ColdFusion (8 and 9) and it works, run it in Railo, and it throws an error. This one took me a while to catch, and once we found it, we realized this code, written in CF6 maybe, has been used in 10+ websites, and all of them needed updated when converting to Railo.

Have you seen the issue yet?

There are 2 dots, instead of 1. 

I don't know why ColdFusion thinks this is valid, maybe because the list contains an empty element with a period being the delimiter or something. I have heard a bit of talk about that dropping empty elements in lists and arrays on the ColdFusion Bug tracker lately, so maybe thats it. Of course, its not valid, but ColdFusion lets it go. At least this works, which I am ok with, but again, there seems to be a lot of things like this… where it seems like it should be caught and throw an error. Of course Railo did, and I had to give the developer who wrote it a bit of grief, its just another example of how Railo is strict, and I think I like that much more than wondering how much of my code is actually doing what I want it to do.

Summary

I am not saying Railo is perfect, there are some other hiccups and issues I found as I migrated sites, and I'm still finding more all the time (you'll see some blog posts on those soon too)… but I wonder if ColdFusion is being forgiving, or sloppy, and if it is good, or scary and dangerous.

What do you think?

PS: I know the easy fix is writing perfect code, but my team and I haven't mastered that, yet.

 

03
March 2014

Gavin Pickin

Setting up my MBP For Mobile Development - IOS Android Cordova Node Ionic

Android, Cordova / Phonegap, Ionic, IOS, Node.js, Tools and IDEs

After months of playing with this, and playing with that, I have a few apps I have to actually get built, wrapped in cordova / phonegap and out to the app store and the many customers, waiting to download them. As usual, I want to document the silly little stuff you do to get things all setup, just in case Android destroys my Mac, I need to set it all up again. I have installed and played with some of these tools before, but I decided to start fresh, so follow along.

First, you have to make sure Node.js is installed, because everything is easier with node and npm… and its the cool thing to use these days. I said that with a little jest, but Node is really cool, and the more I read about it, the more I want to really dive into it, and if you only use it for Package Management with NPM, it is well worth the install. 

Visit http://nodejs.org/ and install Node. 
It is a pretty simple pain free process.

Next, we want to install Cordova, because this is what we are going to use to package our apps together.

$ sudo npm install -g cordova

 

Once we have Cordova installed, lets create our first cordova project, I'm following along with the Ionic instructions, so I'm going to path to /mobileapps/ and then run this command.

$ cordova create hello com.ionic.todo Todo

 

At this point, if you're using a framework, like Ionic, you can copy the WWW files in, and you'll want to initialize git in the folder too, you can setup the remote later.

$ cd hello
$ git init
$ git add .
$ git commit -m "Initial Commit"

 

Now, we want to add IOS and Android Platforms to our projects, before we can do that, we have to make sure the SDKs are setup and ready to go. If you haven't got Xcode yet, go to the App Store, and download it. While that 2gb+ file is downloading, we'll setup Android.

 

Android SDK Setup

The Android SDK Bundle comes with a lot of tools. 

  • Eclipse + ADT plugin
  • Android SDK Tools
  • Android Platform-tools
  • The latest Android platform
  • The latest Android system image for the emulator

You can download it here

 

For the Mac, I download it, unzip it, and then place the whole directory structure where I want it.
Since it's portable ish, I'm going to place it in a top level directory, with my other mobile stuff… at /apps/_tools/

This allows me easy command line access to those items, which I do similarly with /www/_servers etc
So all my apps will be in /apps/ all my websites are in /www/

Once you extract it and move it to the location you want to have it running, open Eclipse, and it will ask you for your Workspace and will detect the SDK based on that location. If you move the whole folder later, you'll have to specify the SDK path again, but it warns you, and allows you easy access to do so, so not a big deal.

Next, we need to make sure our $PATH includes the platform-tools and the tools directory.

On the mac, an easy way to change your $PATH variable is

$ sudo nano /etc/paths

 

Each line has a different path, add more and save, and the system will make sure your $PATH variable has all of the paths. So I append the following.

/apps/_tools/adt-bundle-mac-x86_64-20131030/sdk/platform-tools
/apps/_tools/adt-bundle-mac-x86_64-20131030/sdk/tools

 

CTRL X to exit, press Y to save

Type echo $PATH and you'll they were not added. Do not worry, CMD N for a new Terminal / Console and you'll see they have been added. The system updates the $PATH variable every time a new console is created.

Now, we can run

$ cordova platform add android

 

And we see the following output.

Creating android project...
Creating Cordova project for the Android platform:
Path: platforms/android
Package: com.ionic.todo
Name: Todo
Android target: android-19
Copying template files...
Running: android update project --subprojects --path "platforms/android" --target android-19 --library "CordovaLib"
Resolved location of library project to: /mobileapps/hello/platforms/android/CordovaLib
Updated and renamed default.properties to project.properties
Updated local.properties
No project name specified, using Activity name 'Todo'.
If you wish to change it, edit the first line of build.xml.
Added file platforms/android/build.xml
Added file platforms/android/proguard-project.txt
Updated project.properties
Updated local.properties
No project name specified, using project folder name 'CordovaLib'.
If you wish to change it, edit the first line of build.xml.
Added file platforms/android/CordovaLib/build.xml
Added file platforms/android/CordovaLib/proguard-project.txt

Project successfully created.
Generating config.xml from defaults for platform "android"
Preparing android project

 

Success, ok, now we need to get back and get IOS ready.

Setting up the iOS Platform - Xcode

Its been a while, so hopefully the 2gb Xcode beast has downloaded, search for Xcode and launch it… to begin the Installation Process.

Agree to the Xcode and IOS Sdk Agreement, you'll be prompted for Administrative Access, and you'll have to close down other Apple apps like iTunes etc. Click quit all, and it will install. Quickly and painlessly, you'll see it has installed, and the floating welcome screen should look something like this

Once you have installed Xcode, you can run this command in the project to add the IOS platform to the project.

$cordova platform add ios

 

and you will see this output.

Creating ios project...
Generating config.xml from defaults for platform "ios"
Preparing ios project

 

Great.

  • We have installed Node / nom
  • We installed Cordova
  • Installed Android SDK
  • Installed Xcode for IOS 
  • Created our first Cordova Project
  • Added IOS platform to the project
  • Added Android platform to the project

Busy blog post… but a couple more things, and lets get this app BUILT.
Lets add a couple of plugins - the Device API to query the Device

$ cordova plugin add org.apache.cordova.device

 

and you see this output

Fetching plugin "org.apache.cordova.device" via plugin registry
Starting installation of "org.apache.cordova.device" for android
Preparing android project
org.apache.cordova.device installed on android.
Starting installation of "org.apache.cordova.device" for ios
Preparing ios project
org.apache.cordova.device installed on ios.

 

To access the console.log features in our IDEs, lets add the Console plugin.

$ cordova plugin add org.apache.cordova.console

 

and you see this output

Fetching plugin "org.apache.cordova.console" via plugin registry
Starting installation of "org.apache.cordova.console" for android
Preparing android project
org.apache.cordova.console installed on android.
Starting installation of "org.apache.cordova.console" for ios
Preparing ios project
org.apache.cordova.console installed on ios.

 

All going well, we should be ready, lets try and emulate the project on the android emulator.

$ cordova build android

 

And you will see loads and loads of output. 
The short of it is the last few lines.

BUILD SUCCESSFUL
Total time: 27 seconds

 

Lets run it in the emulator

$ cordova emulate android

 

Of course, if you forgot to setup the Android Virtual Device, you'll get an error. 
Lets do that now.

The easiest way to do it is open Eclipse, and select Window > Android Virtual Device ManagerFrom there, you can click NEW. I selected Nexus7 gave it a name, target, and left the rest as is.

Now, I go back to the command line, type cordova emulate android again, and it launches. 

It doesn't take too long to see the emulator pop up, but it takes a long time for it to load everything, so rest the restroom, grab a drink and something to nibble, and come back to see what magic has happened.

Ok, I'm back… still waiting… but it looks promising, I'm excited.

The Emulator is up, but after a LONG LONG time, its still not launching the app. I can unlock it, click around some, but its busy, and I don't think it's going to ever finish.

Eventually, I kill it.
I do a little research, and realize that too much RAM for your AVD is a common problem.
Go into Eclipse, edit the AVD, and reduce the ram to 512.

Go back and rerun the command again, and hey presto, we have a winner. 
The emulator is up, it launched into the basic app, but we're in business.

Now, I have to get some real apps running it.
Check back for more mobile fun soon.

Blog Search