Migrating Content from ExpressionEngine to Craft

I've wanted to give Craft CMS a go for a while now and figured the best way to do it is with a personal project. My photo blog is in dire need or a revamp and as such I've decided to migrate to Craft. Installing Craft is dead easy and took only about 90 seconds including time to download the zip file.

Before I started working on the design I wanted to be sure that I would be able to easily migrate four years of content so the first step was migrating my Entries. This turned out to be relatively painless with the use of this import plugin.

I first copied all photos from the live site to my upload folder of the new site. Then in the Craft CP click the gear icon on the top right, then Assets, New source and fill in the fields. both the path and url can be set to relative paths, but be sure to not use a leading slash. After saving click the gear icon again and go to the lower left and update asset indexes. A short time later all your images are in the database. You can confirm this by clicking on Assets in the nav.

The next step is to install the Import plugin by putting the files in /craft/plugins then in the CP click gear and plugins and finally install. This plugin only imports content in CSV format. This requires a new template in ExpressionEngine which will need to be set wiht php turned on in order to force the browser to download the file directly.

The php at the top of the template tells the browser this is an excel file and forces it to download the file on browser load

1
2
3
4
5
6
<?php
    header("Content-type: application/vnd.ms-excel");
header("Content-Disposition: attachment; filename=photoblog-export.xls");
header("Pragma: no-cache");
header("Expires: 0");
?>

The next bit of code contains all our fields and how we want them named. This is important to easily map the fields using the import plugin. Line 20 has entry_date with extra spaces this is due to my blog rendering it and not displaying the code correctly - remove extra spaces for it to work. When migrating images, be sure to only include the image name. My images currently reside in a Matrix field {photos} and a file cell. In order to get just the file name it is necessary to use the file tag pair like this {image}{file_name}{/image} where the outer tag pair is the file field/cell name.

7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
<table>
<tbody>
<tr>
   <td>Title</td>
    <td>Url Title</td>
    <td>Entry Date</td>
    <td>Description</td>
    <td>Image</td>
</tr>
{exp:channel:entries channel='photoblog' limit='500' disable="categories|member_data|pagination|comments" status="not foo"}
    <tr>
            <td>Migrating Content from ExpressionEngine to Craft</td>
            <td>{url_title}</td>
            <td>{ entry_date }</td>
            <td>{description}</td>
            <td>{photos}{image}{file_name}{/image}{/photos}</td>
    </tr>
{/exp:channel:entries}
</tbody>

After downloading the file open it in Excel, you may see a warning that the file format doesn't match the extension, open it anyhow. Now do a save as and choose CSV (comma delimited). My research indicated that this extra step is necessary as using the template to save as CSV with content that contains quotes may not work correctly.

Now to import, click Import in the Nav. Fill in the fields as required and upload your CSV file.

Go to the next page where we map the fields to Craft fields. 

Now hit import, wait a few minutes and all of your entries will be imported. To confirm click on "Entries" in the top nav.  Assuming that new content is added to the live site before this build is finished then all you need to do to update is do a new export with only the new entries since the last import and repeat the process. Super easy.

Now it's time to start templating the new site.

Channel Form and Grid/Matrix fields

I was approached to build a site that is essentially a resume builder; Visitors to the site would visit and fill in a multi page form providing the details to build out their resume. I was positive that this wouldn't be an issue with ExpressionEngine as Channel Form makes it easy to create front end forms and Grid is a first party field that would allow me to have groups of fields auto duplicated for sections such as Job Experience or Education.  However this turned out to not be the 100% correct.

Grid does work with Channel form but it doesn't allow for customization on the front end. You can only display a grid field by using the custom_fields tag pair like this:

1
2
3
4
{custom_fields}
	{field_instructions}
	{display_field}
{/custom_fields}

The issue with this is immediately clear in that you cannot control the layout or design of your fields. Also with a responsive site and a seven field grid, the grid is going to break out of the site as it is output as one long table row that is not responsive. Matrix stays in it's container but there is no documented way to display customly. In the screenshot below, The matrix cells and grid cells have the same width. The only difference is matrix allows the use of P&T switch rather than checkboxes

The final code will allow a custom layout like this which will rearrange itself based on the viewport size due to responsive coding.

Meta Q wrote a post in March 2013, Bring Matrix Data Editing to the Front-End, that addressed this issue using Matrix and Safecracker. Safecracker was renamed to Channel form and has had some changes so I proceeeded to use this post as a guide to do what was needed.

Using that article I got everything set up, but it would only display existing data and would not save new fields. After some frustration and help from @Lincolnpixel, who is currently building a similar project I was able to get everything working. The key to success is to not have new fields appear immediately below existing content as in the publish page, but rather to load a new page with new fields and on save go back to the editing page. To do this I used switchee and kept everything in the same template.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
{exp:switchee variable="{pre_seg_6}" parse="inward"} 
{case value="#^P(\d+)$#|''" default="Yes"}
	{exp:channel:entries channel="tool" url_title="{pre_seg_4}" entry_id="{pre_entry_id}" status="not foo"}
		{exp:channel:form 
			channel="tool"
			include_jquery="no"
			include_assets="yes"
			return="tool/page/6/URL_TITLE/ENTRY_ID"
			status="pending"
			url_title="{pre_seg_4}"
			entry_id="{pre_entry_id}"
			author_id="2"
			logged_out_member_id="2" 
			require_entry="yes"
			}
			
			<div class="fields" id="sortable">
				{tool_experience2}
				<div class="field">
					VARIOUS FIELDS
				</div>
				{/tool_experience2}
			</div>
			<div class="buttons">
				<a href="/tool/page/5/{pre_seg_4}/{pre_entry_id}/add" class="btn btn-primary">Add Experience</a>
			</div>
			{exp:low_variables:single var="lv-paging"}
		{/exp:channel:form}
	{exp:channel:entries}
{/case}
{case value="add"}
	<h3>Add Another Work Experience</h3>
	{exp:channel:form 
			channel="tool"
			include_jquery="no"
			include_assets="yes"
			return="tool/page/5/URL_TITLE/ENTRY_ID"
			status="pending"
			url_title="{pre_seg_4}"
			entry_id="{pre_entry_id}"
			author_id="2"
			logged_out_member_id="2"
			require_entry="yes"
			}
		<div class="fields" id="sortable">
			<div class="field">
				VARIOUS FIELDS
			</div>
		</div>
			<button type="submit" class="btn btn-default">Save Changes</button>
	{/exp:channel:form}
{/case}
{/exp:switchee}

As you can see from the above code the first case displays the matrix field for editing based on the url_title and entry_id being passed through a low variable (see my blog post on how I use Low Variables) using preparse. Additionionally it is required to wrap the channel:form tag with channel:entries in order to display saved data. When clicking the "Add Experience" button it goes to the same page but adds a new segment /add/ which then loads up the page for adding new content to the matrix field. There is no need for a channel:entrires on the "add" page as we are creating new data.

To display the content of each matrix row correctly requires poking around the database and getting the column_id for each cell in the Matrix as you are unable to use the cell_name to call the data. You can find the column ids by opening your database and then going to the exp_matrix_cols table and looking at the first column - match that up with the field name in col_name and add your fields like this replacing "x" with the correct id:

1
2
3
4
<div class="form-group">
	<label for="tool_experience2[row_id_{row_id}][col_id_x]">Your Label</label>
	<input type="text" class="form-control" name="tool_experience2[row_id_{row_id}][col_id_x]" value="{company}">
</div>

 On the "add" page the fields need to be called using the same formula above, however since we are creating new rows instead of 
row_id_{row_id} we use row_new_0. Additionally there are two new hidden fields required.

1
2
3
4
5
6
7
<input class="new_row_id" type="hidden" name="field_name[row_order][]" value="row_new_0">
<input type="hidden" name="title" id="title" value="{pre_seg_4}">
 
<div class="form-group">
	<label for="company[row_new_0][col_id_1]">Your Label</label>
	<input type="text" class="form-control" name="field_name[row_new_0][col_id_1]" >
</div>

In the above code replace field_name with the name of your field. The first hidden field creates a new row and saves it when we submit the form with the help of the javascript from the Meta Q article (will post further down). The second hidden field using the url_title to ensure that data is added to the correct entry. In my case this is in segment_4 and is being passed through low_variables as {pre_seg_4}.

I found that date fields using the native date field type were not working as expected so I used plain field types and used jQuery UI datepicker to. To do this you will need to get a custom jQuery UI theme if you don't want your datepickers to look like the publish page datepickers.  Also the regular datepicker shows days and in a resume the date is usually unecessary so my code set's it to pick only the month and year.

HTML

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
<div class="row">
	<div class="col-sm-6">
		<div class="form-group">
			<label for="date_from[row_new_0][col_id_4]">Date From</label>
			<input type="text" class="form-control datepicker" name="tool_experience2[row_new_0][col_id_4]">
		</div>
	</div><!--6 cols-->
 
	<div class="col-sm-6">
		<div class="form-group">
			<label for="date_to[row_new_0][col_id_5]">Date To</label>
			<input type="text" class="form-control datepicker" name="tool_experience2[row_new_0][col_id_5]">
		</div>
	</div><!--6 cols-->
</div><!--row-->

JS/CSS

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
<script>
$(function() {
$( ".datepicker" ).datepicker({
	changeMonth: true,
  	changeYear: true,
  	yearRange: "1970:2020",
  	dateFormat: 'MM, yy',
  	maxDate: "0D",
  	minDate: "-50Y",
  	showButtonPanel: true,
  	onClose: function(dateText, inst) { 
        var month = $("#ui-datepicker-div .ui-datepicker-month :selected").val();
        var year = $("#ui-datepicker-div .ui-datepicker-year :selected").val();
        $(this).datepicker('setDate', new Date(year, month, 1));
    }
});
});
</script>
 
<style>
/* hide calendar and display only month/year*/
.ui-datepicker-calendar {
    display: none;
    }
</style>

The Javascript code, unchanged from the Meta Q article, that creates the new row is here. Lines 18-32 are probably unecessary as we are no long creating new rows on the edit page but rather creating new rows on separate page due to that not working at all.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
<script type="text/javascript">
$(function() {
    $( "#sortable" ).sortable({
     containment: "parent"
    });
});
$(document).ready(function(){
 
$('.fields').on('click', '.delete_row', function(e){
 
  e.preventDefault();
  $(this).parent().remove();
  if ($('.fields div').length < 1){
   $('#new').val('create first row');
  }
});
 
var $clone = $(".fields div:eq(0)").clone().html();
var $button_txt = $('#new').val();
count = 1;
$('#new').on('click', function(e){
  e.preventDefault();
  if ($('#new').val() != $button_txt){
   $('#new').val($button_txt);
  }
  var $counter = count++;
  var $new_row = $clone.replace(/row_new_0/g,'row_new_'+$counter);
 
  $(".fields").append('<div class="field">'+$new_row+'</div>');
});
 
});
 
</script>

Bug Fix

There is one bug fix that will need to be applied to your ExpressionEngine install if you have more than one matrix field in the same channel. This needs to be applied whether both matrix fields are on the same page (on the front end) or not. If you only have one matrix field then it is not necessary.

File: system/expressionengine/modules/channel/libraries/channel_form/Channel_form_lib.php as of EE 2.9.2 line 3257

Method: replace_tag

Find This:

3257
3258
3259
3260
ee()->api_channel_fields->apply('_init', array(array(
    'row' => $this->entry,
    'content_id' => $this->entry('entry_id')
)));

Replace with This

3257
3258
3259
3260
3261
3262
$field_id = $this->get_field_id($field_name);
ee()->api_channel_fields->apply('_init', array(array(
    'row' => $this->entry,
    'content_id' => $this->entry('entry_id'),
    'field_id' => $field_id
)));

Final Thoughts

We now able to view/edit matrix rows as well as add a new row as necessary. Adding a new row is not as elegant as in the backend due to having to load a new page. However that loss of elegance is more than mitigated by the fact that we can now display all our matrix cells as we want them to look and are not limited to the tabular layout/look of the Control Panel matrix/grid.

You could likely use a modified version of this with the first party grid field, but there would be need to experiment with how content is added and checking the database. Until EllisLab adds additional tags to allow customized display of grid fields, the best approach in my opinion is using Matrix with the above code.

My full template with all seven matrix cells is here.

Grunt and Front End Development

I've heard a lot about Grunt.js and Gulp.js over the last year. It's been on my todo list of things to learn. Last week I jumped in with some help from Tad Ward via Skype.

I initially struggled as I had a project I was about to take on that was using gulp and I misunderstood what was needed. Turned out the files I was given were more examples of what was needed rather than me working with the existing code. Once I got that figured out, I decided to keep looking into both Grunt and Gulp. In the end I decided that Grunt looked easier for more.

I had previously installed Node and work in Git and Compass on all my projects so most of the requirements to get running were already set up on my system. Currently I've only got a testing project setup but how I see it working is that I open up a command prompt with Administrator permission (requires administrator permissions on Windows, not sure about Mac) and navigate to the project folder and then type npm init. There there will be some prompts that you need to fill out then you bare project is set.

The next step is to run the command:

1
npm install grunt --save-dev

The --save-dev installs grunt to this particular project. Once this is done, you then install the other grunt tasks that you want to run. The ones that I want to work with are Compass, Uglify, Watch and Concat. Next type the following commands:

1
2
3
4
npm install grunt-contrib-compass --save-dev
npm install grunt-contrib-uglify --save-dev
npm install grunt-contrib-watch --save-dev
npm install grunt-contrib-concat --save-dev

Next up we need to create a gruntfile which loads the commands and tasks that will be run. Through a lot of trial and error my working gruntfile is here:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
module.exports = function(grunt) {
 
// Project configuration.
grunt.initConfig({
 pkg: grunt.file.readJSON('package.json'),
// CONFIG ===================================/
 
	watch: {
		compass: {
			files: ['**/*.{scss,sass}'],
			tasks: ['compass:dev']
		},
		js: {
			files: ['build/js/**/*.js', 'bower_components/*.js'],
			tasks: ['uglify', 'concat']
		},
 
		livereload: {
			options:{livereload:true},
			files:['css/*', '*.html', 'templates/**/*.html'],
		}
	},
 
	compass: {
		dev: {
		   options: {              
		       sassDir: ['build/sass'],
		       cssDir: ['css'],
		       environment: 'development'
		   }
		},
		prod: {
		   options: {              
		       sassDir: ['build/sass'],
		       cssDir: ['css'],
		       environment: 'production',
		       outputStyle: 'compact'
		   }
		},
 
	},
 
	uglify: {
		dev:{
			options:{
				compress: {
					drop_console: false
				}
			},
			files: {
		      'js/main.min.js': [
		      'build/js/*.js', 
		      'build/js/**/*.js'
		      ]
		  }
		},
		prod: {
			options:{
				compress: {
					drop_console: true
				}
			},
			files: {
		      'js/main.min.js': [
		      'build/js/*.js', 
		      'build/js/**/*.js'
		      ]
		  }
		},
 
	},
 
	// concatenate js and output unminified.
	concat: {
	    options: {
	      separator: '\r\n\r\n /******/ \r\n\r\n',
	    },
	    dist: {
	      src: ['build/js/*.js', 'build/js/**/*.js'],
	      dest: 'js/main.js',
	    },
	},
 
 
 
});
 
// DEPENDENT PLUGINS =========================/
 
grunt.loadNpmTasks('grunt-contrib-watch');
 
grunt.loadNpmTasks('grunt-contrib-compass');
 
grunt.loadNpmTasks('grunt-contrib-uglify');
 
grunt.loadNpmTasks('grunt-contrib-concat');
 
 
// TASKS =====================================/
 
grunt.registerTask('default', ['compass:dev' , 'uglify:dev' , 'concat', 'watch']);
 
};

What happens here is that the watch command Sets up files to watch and tasks to do. First it runs compass so that my Sass files get processed and output to the folders indicated in the Compass command below. Next in Watch js is uglified, i.e. compressed and concatenated. Lastly I use Concat to concatenate all the js files and output a version that is not minified. The minified version will be used in production. Line 76 adds line breaks between concatenated files for easier readability.

I do a lot of work for other agencies and feel that I should also provide them with a readable version of the javascript which this accomplishes. Now grunt outputs a main.js and a main.min.js file to the /js/ folder.

Back to the Watch command on line 18 we see:

18
19
20
21
livereload: {
	options:{livereload:true},
	files:['css/*', '*.html', 'templates/**/*.html'],
}

This is pure gold. Before I got this working I had gone after a red herring, grunt-browser-sync, which would never install and led to much frustration. Once I found the docs for Live Reload and set it up to reload css/js/html on change I was set. This command actually watches the CSS file for change. So once compass processes the Sass files and outputs the new CSS file then the browser reloads. There is a slight delay, but it is nice not having to manually refresh the page. To get it to work on your local files you need to either add a bit of javascript to the site or use a browser extension (Safari, Chrome, FireFox). Now all I do is make a change to the sass files, wait a second and the change is reflected on the site locally. No need for me to manually refresh the browser. This is brilliant. *UPDATE* - changed files for livereload and now works when developing in ExpressionEngine. I have my templates folder at public_html/templates - the /**/ is needed so that it will look for all html files inside all subdirectories.

When it's time to start development in the in the Command Prompt all you do is type grunt and then it performs the tasks set up on line 101.

101
grunt.registerTask('default', ['compass:dev' , 'uglify:dev' , 'concat', 'watch']);

Now when I'm ready to move the code to production I can run different commands for example grunt compass:prod will run the compass task but instead of adding line comments and keeping the css readable everything will be compressed with no comments. Additionally running grunt uglify:prod will take care of the js files and remove all console.logs, which are not really necessary on production.

Assuming you have a number of tasks that you like to use on every site, this whole process will be even easier. All you need to do is copy package.json and gruntfile.js of an existing project into a new project folder. Then open up your command prompt with administrator permission and navigate to the new folder and then enter this command:

1
npm install

Voila everything is installed and you are set to start developing. Comments and suggestions on tasks that may be useful to me or better approaches are appreciated.

Front End Syntax Highlighting With ExpressionEngine

Yesterday I was asked via twitter how I do syntax highlighting on this blog. It's actually quite simple with a couple of addons. I was using a different addon, but it stopped working when I upgraded the site last year and then switched to using EE Syntax by Eric Lamb Which does the bulk of what is needed.

My blog channel has Three custom fields - a blog image (file), a copy field (wygwam) and code sample (matrix). The matrix field contains all code blocks and then using the approach outlined here I place the code blocks into the wygwam field. I chose to use MX Jumper, but the other options will work just fine - this could also be done using Stash.

Not sure why, but when I try to post the template code that renders the template code the page doesn't load. Have tried several workarounds, but no go. Can post any code except the code that displays code. Instead check this pastie.

Your Voice Here
Leave a Comment