Blog

Blog Archives - 15 Record(s)

Year: '2014' - Month: '2'

12
February 2014

Gavin Pickin

CFML Server - A Different type of ColdFusion Repo - ColdFusion Installs

CFML Server, ColdFusion Install Repo

UPDATE: New home for the CF Install Repo Available: http://www.gpickin.com/index.cfm/blog/coldfusion-install-download-repo-new-home-for-all-your-files

Today, Adam Cameron posted an interesting little blogette about things he is not
The short of it all, was that recently a lot of people have been asking for older versions of ColdFusion, and as Adam put it, he is not a download service, and its not his job to look after everyone else's software. 

This isn't the first time I've heard about it lately, and I even put a copy of one or two downloads on a server for someone to download, from a collection I have laying around on some Backup Drives here and there. I thought it would be cool if there was a centralized shared location that anyone from the CFML community could go and get a ColdFusion Installer, without having to Login to Adobe, and realize they only have the latest 2 versions available (See Note Below). With CF 11 coming closer and closer by the day, CF9 will no longer be available sometime soon, so I guess now is as good a time as any to start this repo myself.

I had thought about how we should do this repo for a little bit now, thought about servers, free space etc, and then today when I read Adam's article, I thought why not just use Copy.com. Copy.com is a pretty cool service, much like DropBox, but a couple of key differences

  • You start with 15gb
  • You get 5gb per referral (use the link below so we both get 5gb for the referral)
  • Shared Folders Storage Amounts are SHARED between the SHAREEs, so if you have 2 users, you get usage for only half of the shared files, share it with 10 people, you get 10% each, compared to Dropbox, if you have 10 users sharing, you each get charged 100% of the storage amount.

I signed up an account specifically for this project, and created and shared a folder called ColdFusion Installs.
I decided to sort the installs by Operating System, and then 32bit and 64bit.

I happened to stumble across one of my old CDs, which happened to have ColdFusion 5, ColdFusion MX 6.1 and ColdFusion MX 7 for windows, so I thought it was pretty cool to start off the repo with some oldies. I even have the serial numbers still, but I did not share those, find your own. 

I also uploaded a series of other installs I had on my local machine(s), I will look through my Raid Backup later when I get time, but when I checked back on Adam's blog, I noticed another interesting post.

David Epler said… 

Or just follow the links and download direct from Adobe still.
http://www.dcepler.net/post.cfm/last-day-to-download-cf-8-and-9-from-adobe-with-verity

I read his article, and he was right, before Verity was removed from ColdFusion 9, while ColdFusion 8 was still available, they had a series of links on their page. Through some sneakiness (you'll have to read his blog to see what I mean) he acquired the names of the files, hashes, links etc. The file were supposed to be only good until a certain date, but all they did was remove the links, so all the links to the files on Adobes Servers are still active.

NOTE - You have to follow his instructions, and login to Adobe's site first, otherwise the links will return a 404 error. 

That was a great find, and one I had not ran across in my travels, so I have included a read me file which has the link to his blog (shown above) and a little information… so you can get the downloads direct from the source
I will also place them in the Shared Folder for ColdFusion Installs, just in case the links do end up dead one day.

If you have any files, and want to help out, let me know.You should be able to place files in the share, if you cannot, let me know, and we can figure it out. I haven't done too much with Copy.com sharing yet.

The referral link to get your 15gb + 5gb bonus
https://copy.com?r=p7GPUQ

The link to the shared folder
https://copy.com/i0LSL1n8A6QC or http://tinyurl.com/cfdownloads - thanks to Charlie Arehart for the tinyurl. [EDITED]
Update: Use Link At Top of Page

 

We have already helped one person who was chatting with me on Twitter, hopefully it will become useful for others looking for installs for testing, compatibility or just plain curiosity.

Bonus: I have access to these from anywhere and I don't have to remember which CD case its in, or which backup drive I put it on. 

 

Thanks for reading,

Gavin

10
February 2014

Gavin Pickin

Online Interactive Learning - TryCF - Tutorial Contest Over

CFML Language, Online Interactive Learning

Time to talk about one of my favorite new and cool Online Learning tools around, and yes, it's a ColdFusion tool, TryCF. I have talked about TryCF in earlier posts in my Online Interactive Learning category, and one post I talked about submitting my first tutorial. My tutorial was one of the first submitted and approved by the community, and it qualified for the tutorial contest.

Tutorial contest, that sounds interesting. You're right, TryCF created a contest to ask the community to help create some tutorials to help get the site loaded with a variety of topics, from a variety of authors, to help a range of new comers to the language get a start. Adobe even threw their hat into the area and supported the contest, giving all accepted tutorial authors a CF Builder License, and the winner, a Visa Giftcard for $300.

Abram, the creator of the site, created the site to help his Son Sabin learn ColdFusion. His son Sabin had done well with other tools in the market for other languages, like CodeSchool, LearnStreet, but ColdFusion was missing this type of tool. The contest was designed so Sabin would be the judge. Its a very well featured and functional sites, and unlike a lot of Open Source and Free Education tools out there, its a pretty slick looking site. It runs on Railo and will run on Adobe ColdFusion eventually too, and the site uses jQuery and Angular. The best part of the site, is the built in Editor, which you use in the tutorials, and you can even use it as a scratch pad, and load Gists and edit and run them too... you have to check it out.

Abram mentioned there were over 30 serious tutorials submitted, and I must say, I was proud to be one of the "final 3". One of my friends Scott Steinbeck (a former colleague) was also selected in the "final 3" with 1 of the 3 tutorials he submitted.

I won't spoil the mystery, so if you want to know more about the contest, you'll have to read the article (the link is all the way at the bottom)

Please look over the site, and take some time to think about writing a tutorial to help the community.

There are some good tutorials now, but there is a lot to learn about our great language, and the best thing about it is our community, so please do your part and support it.

Thanks,
Gavin

Ok, you can read it now.

http://www.trycf.com/blog/And-the-winner-is

05
February 2014

Gavin Pickin

Amazing Coupons, Discounts, Freebies and Value Galore - cf.Objective() and Into the Box

cfObjective, Conferences, Steals and Deals

In the ColdFusion world, conference season is just around the corner, with cf.Objective() in May, with Into the Box the day prior, the time to start planning and buying your tickets is now… before all the specials and deals end.

cf.Objective()

cf.Objective(), the Worlds Biggest Enterprise CFML+ Conference, packed with 5 amazing tracks, and 3 solid days of amazing content, is coming to the end of its Early Bird pricing.

If you book on or before Feb 14, 2014, you will get the early bird price of $849 (749 if you are an Alumni, User Group Manager or Government)

( Information taken from cf.Objective() website )

  • 3-4 People 5% off
  • 5-9 People 10% off
  • 10-14 People 15% off
  • 15-19 People 20% off

Your registration fee includes:

  • Access all Conference Sessions
  • Power Access in the Meeting Rooms
  • Breakfast, Lunch and two breaks during the day. Dinner is on your own
  • Free Wireless Internet in meeting & sleeping rooms
  • Tuesday Night Welcome Reception
  • Birds of a Feather and Lightning Talk Sessions
  • Official cf.Objective() 2014 Swag and conference program guide

cf.Objective, the only enterprise engineering conference for ColdFusion is your connection to:

  • High quality sessions offered by the leaders in the industry
  • A 5 to 1 ratio of attendees to speakers, giving you direct access to leaders in the field to discuss their development challenges
  • You have a chance to talk to the people who are the leaders in the field about what every challenge you are having with your application or project
  • An intimate feel. If you want to get lost in a crowd, this conference is not for you!

 

Into the Box

Into The Box is a full day 2 track conference on May 13, the day before cf.Objective and is jam packed with an amazing amount of value. If you purchase your cf.Objective() tickets, you get a link to buy your Into the Box ticket for $50 off the normal price of $199.

  • You can get 20% off for groups of 3+
  • You can get 20% if you are part of a non-profit
  • You can get 50% off if you are a student

But wait, there's more

Your entry ticket also gets you 

And they are working on more giveaways, you'll have to attend to find out all the goodies.

But wait, there's more

Plus Breakfast, Lunch, and Happy Box - which will include Raffles, Ortus Team Q&A, free food, free drinks, free live music while you wait for the cf.Objective() Registration to open.

If you still don't think you can convince your boss, check out the "Justify Your Trip to Into The Box" pdf to give to your boss / manager.

4 days of amazing content, great people, great presenters, loads of value… plus you'll get to see me, reason enough right?

You can read more of my cfObjective Posts here, Why I wanted to speak at Into the Box here.

Sign up today... and I'll see you there

http://cfobjective.com/

http://www.intothebox.org/

 

Don't wait too long,

Gavin

04
February 2014

Gavin Pickin

Online Learning - Podcasts - Mind Opening Interview with the Discoverer of JSON Douglas Crockford

Online Interactive Learning, Podcasts

I am a late comer to the world of Podcasts, but with the birth of my son, I had a lot of time holding babies, feeding babies, changing diapers, and podcasts helped me fill the time and getting my learning kick at the same time. I listen to several podcasts, and the good ones, I zap back hundreds of episodes, to soak up each and every ounce of value out of them. I will not make a post for every good episode I listen to, but if they really stand out, I will post about them here, and invite you to listen, and see what you think of them.

Today, I listened to one of the Hansel Minutes shows, episode 396 actually. In this episode Scott Hanselman interviews Douglas Crockford at the AngleBrackets conference in Las Vegas. Douglas Crockford is the "discoverer" of JSON, as he states it was always there, he just uncovered it, not invented it. He also build JS Lint, and have a book called JavaScript - The Good Parts

In this interview, a lot was discussed, including how we have forgotten our roots in the software world, how our software legacy can help us as we progress in the world of software today. The concept of how our tools are not helping us, but crippling us in our goal to create bug free software that solves a business / organization problem. 

It really made me think, as the ideas are really against the flow of what normal developers think and believe in. We are not our code, and sometimes we take an emotional response to our code, and our languages, and if we sit back and think, really think, we might see the world of software in a different light.

A snippet from Scotts About Us Page

My name is Scott Hanselman. I'm a programmer, teacher, and speaker. I work out of my home office in Portland, Oregon for the Web Platform Team at Microsoft, but this blog, its content and opinions are my own. I blog about technology, culture, gadgets, diversity, code, the web, where we're going and where we've been. I'm excited about community, social equity, media, entrepreneurship and above all, the open web.

Check out the episode. I think its well worth the listen. 
http://hanselminutes.com/396/bugs-considered-harmful-with-douglas-crockford

Enjoy,

Gavin

03
February 2014

Gavin Pickin

Dev Ops - From Mail Logs to DB Stats in a CFML Dashboard - Part 4

CFML Language, Server Admin

Welcome back to the fourth post in this mini series on how to take some Mail Logs from our qmail servers, and massage them and move them around, and finish with them on a cfml box ready to massage them a little more, and make some pretty and readable graphs and tables, to keep our Mail Servers in the good books, and off those pesky Spam Blacklists. In post one, two and three, we found the files, identified what we wanted out of the log files, planned out how we were going to use cron to move files, reset perms, and finally grep the important stuff out of the logs, and get it ready for our cfml engine to do its thing.

Today, we're going to look at the start of my scripts (rusty at first, I admit), that read in the log files, and import them into the database. As I mentioned in previous posts, I think I have all the data out of the logs that I need, although, I bet I haven't found all the ways I will want to use this data, so I decided to import the data in raw format for step 1, and step 2, I will crunch the raw data to pull out the records of use. By leaving the raw data, with some vitals information along side that, like Date Time Stamp, I can always go back and crunch that data for another pattern, when I find it in the future, and by storing the Date Time Stamp, I can of course purge old Raw Data as time passes, and I care less and less about it.

To get my scripts up and running, I'm starting in a folder, with some plain and simple single files, no framework, no real organization, just single pages of code, while I experiment and get things working. There are some serious duplication going on, which I will refactor out in time, so I am sure you will see a million things wrong with the code, but try and think about what I'm doing, compared to how I'm doing it. Although, of course, we will refractor this code, and if you want to give advice, please do, just letting you know its raw at this point, and I expect a lot of it to be refactored into functions etc.

As I build this, I am aware that cfml as a language is not built for reading logs and manipulating, although, I think it does pretty well, but to keep track of it, I'm going to throw in some timers, to keep an eye on how things take. Like most people, we usually knock up our own little tick debugging notes, and its pretty effective, but I decided to try out my buddy Adam Cameron's little stopwatch function he wrote about a while ago. It does the same job, but uses an object, stores the information, and makes it easy to add laps, and then dump the whole object when you're done. It takes care of total time, incremental time, and is simple to use. Why invent the wheel, right… so here is the link to his little stopwatch… and I'm including it in the header of my file.

<cfsetting requesttimeout="600">
<cfinclude template="stopwatch.cfm">
<cfset ds = "dev_mydatasource">

<cfparam name="thelogname" default="clean_maillogH.log">
<cfset stopwatch = makeStopwatch()>

 

First, I give the file a nice long timeout, knowing this will be a pretty long running task. I include the function for the stopwatch, and then set my datasource. Since I'm going to be working on 2 types of files, I want to pass that in, I'll default it to the hourly log file, and then create an instance of the stopwatch to use on my page. No rocket science there.

<cfset stopwatch.start("Begin timing")>
<cfquery name="ind_control" datasource="#ds#">
select * from ind_control
where filename = <cfqueryparam value="#thelogname#" cfsqltype="cf_sql_varchar">
</cfquery>

 

Next, I start my stopwatch, labelled "Begin timing" and then query my control table. My control table is pretty simple, I have the following fields 

  • id - primary key auto increment
  • filename - name of the file, differentiates between daily and hourly log
  • dte_good - date of the first record in the file
  • int_good_end - last record processed in the file
  • int_endoffile - was the end of the file reached

Since I have 2 log files I'm working on, I have 2 rows in my control table. One for clean_maillog.log for daily, and clean_maillogH.log for hourly. As I process records, I am going to check the date of the first entry, if the date matches the date in the log file, we will continue processing from where we left off, the int_good_end value. If the date does not match, the file is a new file, and we must start processing from the first row. End Of File is recorded if we reached the end of the file in our last batch of processing. In the hourly, this will be set hourly, but if the process runs, and finds the the last row processed is no longer the end of the file, this end of file flag is reset, and processing will continue as it did previously.

To show you how good cfml really is with reading files like this, we'll build the file and show you a few numbers as we go… so first, lets read our file, and count the number of lines in the file.

<cfset thecounter = 1>
<cfloop file="/home/maillogagent/#thelogname#" index="thelog" >
    <cfset thecounter = thecounter + 1>
</cfloop>
<cfoutput>#thecounter#</cfoutput>
<cfset stopwatch.lap("Counter Loop done")>
<cfdump var="#stopWatch.getTimeline()#">

Running on the current file at the time of writing this, 132322 lines

    It takes 219 milliseconds to increment the counter, and output the counter at the end

    It takes 238 milliseconds to increment the counter, and output the counter for each line and at the end.

That is pretty quick.
Lets try a REALLY big file, and see the difference.

One of the raw, ungrepped files reaches 1.179gb… lets see what our cfml can do with that.

That is almost 10 million records, and it takes under 10 seconds to read and loop through every line, incrementing the counter, and outputting it. The worse week we have had in the several months I have been looking at logs, hits a peak of 400k, and that was with a huge outbreak of mail…. so in the scheme of things, cfml can read the files, and read them pretty quick… on average about 1 second per million lines, without any processing… so lets see what happens when we want to do some basic manipulation and preparing for database activity.

We're going back to our weekly log, of approx 132000 records for the next set of tests.Now, inside the loop, we're going to pull out the date, and create date time objects for each line. When we go to pull the actual data in, we'll pull it in in batches, but for now, for timing, lets see how this builds out.

<cfif find(' independence ', theline) gt 0>
    <cfset cleanedup_val = left(theline, find(' independence ', theline)-1)>
<cfelse>
    <cfset cleanedup_val = left(theline, find(' new-independence ', theline)-1)>
</cfif>
<cfset thedate = parsedatetime(dateformat(now(), "yyyy") & " " & cleanedup_val)>
<cfif thedate gt now()>
    <cfset thedate = dateadd("yyyy", -1, thedate)>
</cfif>

 

First, we pull the first piece of the line off, as it contains the date. We have to use an if statement to find that, as our server name changed as we migrated servers, so some log entries have independence as its name, and others as new-independence… so we find the location of that string, and grab everything to the left of it.

We then add the year to the string, since it didn't have it in the logs, and then parse the concatenated string to get use a date time object. Of course, if I parse anything from Dec 20th, and put the 2014 in front of it, the time is wrong, so if the date is greater than now, the log must be last years, so we just take a year off of it, and we have our date object ready for manipulation.

Lets run our test and see how long the read and date parsing takes.

So cfml loops through a file with 132000 lines, parsing the date on each one, and we're still under 1 second. So far so good, of course, the more we do, the more overhead we're adding to each and every line. Next, lets add a database call to the mix.

Each and every line, we're going to check to see if that date exists in the database already.

<cfquery name="logCheck" datasource="#ds#">
select id from ind_log
where dte_log = #thedate#
</cfquery>

 

Now we're getting into some big numbers, 132000 database calls, across the network to our database server, and this is where you see the bottleneck. Trying to process 132000 records with 1.6 millions records in the table (I have been loading data as I write this) and the request times out, at the servlet level. I assume there is a timeout which I have no overridden in Tomcat, and 132k does not success. I set an if statement, only running the query for the first 100,000 loops, and these are the results

To see how much affect the data has on the timing, I duplicated the table, with only a few records in it, and ran the same code, and these are the results.

So 39512 / 100000 = 0.395        with 1.6 million records to be searched (with an index on the field)

Vs 55188 / 132332 =  0.417      with only a handful of records

Seeing there was not much of a difference in the results, I decided to run the tests again, and interestingly enough, against 1.6 million records, and it completed in 52 seconds, so it looks like my initial test that failed was because of another task, job, or external factor affecting the speed, and putting the request over that 60 second time limit I believe is imposed at the servlet level on my server right now.

The point is, we can crunch quite a lot of data with cfml, and even with 100k-130k of database queries (albeit reads) we can get a lot done.... not too bad I think for a language where this isn't its strong point, but it does enough to be useful in these situations, especially for what I'm doing here.
I think we have covered a lot, next post, we'll continue processing our logs, and look at insertion speeds, and decide whats a safe and logical batch size. 

Thanks,

Gavin

 

 

Blog Search