A question I get a lot is how to solve date formatting issues in CMSImport. Unless you are using SQL Server all imported values are of type string. So when you try to import date values the data layer will try to convert the value to a DateTime object, if it can’t parse the string you will see an error. These errors can be solved. Let’s take the following example from a Wordpress export file that can fail.

          
            <
            wp:post_date>2011-11-03
17:29:18</wp:post_date>
        

When mapping the value in CMSImport you can select the advanced mapping option, this will show the following mapping options if you map against a DateTime Property. As you can see you can specify a date format. Check the custom date formatting section for the exact formatting syntax. When you specify a format that exactly matches the string from your datasource(TIP use the refresh option to update the preview) CMSImport will use that format to convert the string to the DateTime object and all will import fine.

Hope this helps you!

A few weeks back  I’ve released CMSImport 2.1. Apart from the awesome Logo made by Arnold Visser I’ve added some functionality as well.

Import dashboard for Content Editors

CMSImport is great for developers. They can create import definitions and execute, or schedule  them. When you want to allow your content editors uploading new data and import that data you need to allow them access to the developer section, or let them upload a file to a specific location and schedule a task that imports that file very hour or so. Well not anymore. Now  you can use a dashboard control and allow users to import their data using that dashboard control. The user can select an import definition, upload a file containing the data to import and optional upload a zip file containing the media files. Before importing the data will be verified to ensure all columns still exists in the new uploaded data file. This feature is a direct request from customers who use CMSImport to update their TeaCommerce or uWebshop catalog, but can also be used for any other import requirement of course.

As a developer you still have full control what you want your users to import. Using the ImportDashboard settings page you can enable/disable Import definitions you want your end users to use. You can only select import definitions that allow upload of a file such as Excel, Csv, Xml etc. You will also see an xml Snippet you can customize to your needs and paste into the dashboard.config file. Using the access element you can also restrict which User types can access the Dashboard control.Check out this wiki for more information about configuring Dashboard controls.

Finally support for Media using MNTP

With the 2.1 release it’s finally possible to upload media using the Ucomponents Multi Node Tree Picker. When you select advanced mapping options when mapping against a property based on the MNTP datatype you will see the following options screen. Set the media location where to store the media and files will be imported automatically. The benefit MNTP has over normal Media pickers is that you can use a comma separated string of media files you want to import instead of just one file. When mapping against a MNTP datatype you can only set media settings if the datatype is configured as a media picker, otherwise you will get a message that it is not possible to set the advanced options.

A question I get asked a lot is if it is possible to assign a template based on a datasource value. The answer is yes, normally the default assigned template will be assigned when importing your data and this is what 99% of the users want and expect. But using the RecordImported Event we can get the datasource value from the database and use that alias to map the template against the imported document.

Example

In this example we have some data coming from a database that we want to a standard Runway text page. Some items such as Integer and Donec are really important and we want to display these items in Pink. In Umbraco we added a new template called PinkTemplate and as you can see in the image below we added that Template alias in our Datasource that we are going to import.

As I mentioned earlier we can use the event system to manipulate the document during import . You can use the event system for CMSImport in the same way you use the normal Umbraco event system. In the example below you see that we derive from ApplicationBase and wire up the RecordImported event in the constructor. Since we only want to set the template during the initial import, we first check if the Importaction is  ImportAsNew. Then we need to check if the template column from the datasource is not null. You can access all  items from the datasource  using the e.Items collection. When the template column is not null, we get the template id from the database and assign that value to the document.

             1: 
            using CMSImport.Extensions.Import;

             2: 
            using umbraco.BusinessLogic;

             3: 
            using umbraco.cms.businesslogic.template;

             4: 
            using umbraco.cms.businesslogic.web;

             5:  

             6: 
            namespace AssignTemplate

             7: {

             8: 
            /// <summary>
          

             9: 
            /// This class wil automatically
assign a template to the document when being imported for the first time using the
RecordImporting Event
          

             10: 
            /// If you want to use this
class in your own project simply rename e.Items["Template"] to the template
column you use
          

             11: 
            /// in your datasource
          

             12: 
            /// 
          

             13: 
            /// This code will work in
both the Free and PRO edition of CMSImport and is written for version 1.2. 
          

             14: 
            /// If you have any questions
please use the forum or email support@soetemansoftware.nl 
          

             15: 
            /// </summary>
          

             16: 
            public
            class AssignTemplateUsingDatasource
: ApplicationBase

             17:  {

             18: 
            /// <summary>
          

             19: 
            /// Initializes a new instance
of the <see cref="AssignTemplateUsingDatasource"/> class.
          

             20: 
            /// </summary>
          

             21: 
            public AssignTemplateUsingDatasource()

             22:  {

             23: 
            //Whenever a records gets imported 
          

             24:  ContentImport.RecordImported += new ContentImport.RecordImportedEventHandler(ContentImport_RecordImported);

             25:  }

             26:  

             27: 
            /// <summary>
          

             28: 
            /// When a record got imported
this event is raised.
          

             29: 
            /// </summary>
          

             30: 
            /// <param name="sender">The
imported document.</param>
          

             31: 
            /// <param name="e">The
<see cref="CMSImport.Extensions.Import.RecordImportedEventArgs"/>
instance containing the event data.</param>
          

             32: 
            private
            void ContentImport_RecordImported(object sender,
RecordImportedEventArgs e)

             33:  {

             34: 
            //Only assign the value when
its imported for the first time
          

             35: 
            if (e.ImportAction
== ImportActions.ImportAsNew)

             36:  {

             37:  Document doc = sender as Document;

             38: 
          

             39: 
            //Get the template value from
the datasource. Datasource values are stored in the Items event arguments
          

             40: 
            //Replace templateName with
your column name in the datasource!!!
          

             41: 
            string templateAlias
= e.Items["templateName"].ToString();

             42:  

             43: 
            if (doc != null &&
!string.IsNullOrEmpty(templateAlias))

             44:  {

             45: 
            //An alias is specified, try
to get the template based on the specified alias 
          

             46:  Template template = Template.GetByAlias(templateAlias);

             47: 
            if (template.Id != null)

             48:  {

             49: 
            //Template is found assign
it to the document
          

             50:  doc.Template = template.Id;

             51:  }

             52:  }

             53:  }

             54:  }

             55:  }

             56: }

When the import is finished you see that all items are imported and when you we take a look at the Integer document you’ll see that the assigned template is PinkTemplate instead of the default Runway Textpage template.

Why isn’t this supported in the UI?

As I mentioned before most people expect to assign the default value. When I include the assign template via datasource functionality in the UI it’s an option that you need to think about before setting or ignoring the option. My idea with CMSImport is that I want it to be a no brainer to use and introducing this option will make it a little bit harder to use. So if you want this I think it’s better to write 10 lines of code instead.

Downloads

Below you can download the complete source code. Change e.Items[“templateName”] to the column name you want to use in your project, build the project and add the DLL to the bin folder of your Umbraco install. That’s all!

Download the source code here

Last week I’ve released CMSImport 1.2 (Pro). This was a minor release mainly to fix a few bugs

Change log:

• Fixed string delimiter for csv files
• Field adaptors on default properties didn’t get hit
• Throw error when login is null for member import, otherwise you can’t create new members and Umbraco will throw errors on you.
• Introduced a new FieldAdapter that can map user logins against a Member picker data type. CMSImport will auto convert this to the corresponding member id.
• Tested against Umbraco 4.6 (beta)

Again this was a very small release. Mainly to fix some errors.

CMSImport 2.0

The most requested feature is structured import. I’m very happy to announce  that this will be included in CMSImport Pro 2.0. The video below demonstrates how this will work. I've tried to keep this as simple and flexible as possible and hope I have succeeded, if not I hope to hear from you.

CMSImport 2.0 Structured import demo from Richard Soeteman on Vimeo.

Other new functionality for 2.0

Of course structured import will not be the only new feature. Below you find a list of all other functionality that will be included

Support for more Data sources

CMSImport 2.0 will support new data sources to import from such as:

• Rss
• BlogMl
• Wordpress export
• Excel
• MS Acces

If you have a specific product, or data format you want to import from please let me know through the comment form.

Configurable field adapters

In version 1.1 I’ve introduced field adapters. These field adapters are great to convert data from a data source to the format that is accepted by the Umbraco data type, but sometimes you need to specify some additional information. For example when you import data and want to map url’s against the Ucomponents Url picker you want to specify if the link needs to be opened in a new window or not. Currently that is only possible with a config file, not very friendly. In 2.0 it will be possible to specify some additional options like you have to do now when you import related media.

Support for third party packages

CMSImport 2.0 will come with support for third party packages such as UComponents, SuperSimpleWebshop,Teacommerce and Blog4Umbraco.

I’m sure there will be extra functionality added along the way.For now I hope you like what you’ve seen in the video and read in this post.

Ok official it was  released in the beginning of October, but the manual wasn’t finished and I couldn’t upgrade existing clients to 1.1. From a customer satisfaction point of it was very important for me to have these things ready before blogging about all the cool new features. So what’s new?

Media Import

The main feature that is added to CMSImport PRO is the ability to import media related to content or member data. This isn’t a separate import process but integrated in content or media import. When CMSImport finds a reference to a relative path it will try to get the item and convert it to a media item, or store it in the media folder in case of an upload field. The only required thing is that the original media folder is copied to the root of your Umbraco folder.

In the example below the img folder of the original site containing two images is stored in the Umbraco root.

In the Import wizard you can specify that you want to import media items also. Check the option “Import content related media”. If you want to map media against media pickers and/or the TinyMCE editor, you need to specify a media folder also. CMSImport will keep the imported folder structure.

Then in the next step you can create the mapping like you would normally do.

1. When a reference to an image is found in the content, CMSImport will create a media Item and update the image source to the new Media item(Currently only images are supported in Rich Text. Version 1.2 will have support for files also. Media Pickers and Upload fields will work with files also.)
2. When an image (could also be a file) reference is mapped against an upload field. CMSImport will store the image in the Umbraco Media folder and update the reference in the Upload field
3. When an image (could also be a file) reference is mapped against a media picker. CMSImport will create a media item and store the Id of the media item .

When the import process is finished you’ll see that the media items are imported

And when you open an imported item in the content section you’ll see that all the references are updated to the imported media items

\

Again you don’t need to do anything extra apart from downloading the original media folder to you Umbraco Install.

Currently this import process will work for the following datatypes:

• Upload Field
• Media Picker
• TinyMCE (Rich text editor)

In future releases CMSImport will support custom datatypes and File import for the TinyMCE datatype also.

FieldAdapters

FieldAdapters are used to convert original values from the datasource to a value the DataType expects. In previous versions when you wanted to import boolean data from a CSV file and map that against a true/false dataType, it would have failed because the True/False datatype could only handle the integer values 1/0 or a real boolean.

FieldAdapters fix those issues. During import of a field CMSImport will check based on the underlying datatype if a value can/must  be converted.

The following snippet contains the implementation of the BooleanFieldAdapter( just to show how little coding is needed for a better import experience).  It will add a relation to the True/False dataType based on the DataTypeID property . This must be the same GUID as the True/False datatype. Then in the Parse method  it will inspect if the value is either 0 or 1, if not it will convert the value to a real boolean.

          
  1: publicclass BooleanFieldAdapter
: IFieldAdapter 
          
  2: {

          
  3:     ///
<summary>
          
  4:     ///
Contains the GUID of the true/false datatype we want to parse using this FieldAdapter
          
  5:     ///
</summary>
          
  6:     ///
<value></value>
          
  7:     public Guid
DataTypeId 
          
  8:     {

          
  9:         get { returnnew Guid("38b352c1-e9f8-4fd8-9324-9a2eab06d97a");
} 
          
 10:     }

          
 11: 

          
 12:     ///
<summary>
          
 13:     ///
Parse the data
          
 14:     ///
</summary>
          
 15:     ///
<param name="value">The value to parse</param>
          
 16:     ///
<param name="fieldAdapterService">The field adapter service.</param>
          
 17:     ///
<returns></returns>
          
 18:     publicobject Parse(objectvalue,
Property property, FieldAdapterOptions fieldAdapterService) 
          
 19:     {

          
 20:         if (!(value.Equals("0")
|| value.Equals("1"))) 
          
 21:         {

          
 22:             //
Parse boolean value and return result
          
 23:             bool boolValue
= false; 
          
 24:             bool.TryParse(value.ToString(), out boolValue); 
          
 25:             

          
 26:             return boolValue; 
          
 27:             

          
 28:         }

          
 29:         returnvalue; 
          
 30:     }

          
 31: }
        

This mechanism is very powerful. In fact the whole media import is based on this. Currently it only support a few default Umbraco Datatypes. For later versions I will make sure that FieldAdapters will fix import issues for all popular third party datatypes, like Ucomponents also. If you don’t want to wait for that use the API to create your own FieldAdapter and include it in your package.

Improved API

In 1.0 you could extend CMSImport. but extension points existed in different assemblies. With version 1.1 all extension points exist in the CMSImport.Extensions.DLL. Using this assembly you can create your own DataAdapters and FieldAdapters and you can use the event system. In chapter 8 of the manual you can read all about this. I’ve also created a sample project that you can use as a start. This contains samples from how to create a DataType to how to support Structured Import using the Event System. This was written for CMSImport PRO but you can use it for CMSIMport also, although not all samples will work. The sample code uses the AdventureWorks database a  DataSource. You can download this database from  CodePlex and you can download the Samples from the CMSImport site.

Small other changes

This following other small changes are included in this release.

New version of the CSV Parser

In the 1.0 release I was using an older version of the this Csv Parser. This one is upgraded  to the newest release an didn’t have any issues with it.

Better Error Handling for Saved Imports

When you deleted the root folder in CMSImport PRO 1.0. The only message that you would get was “Object reference not set to an instance of an object” and you could start all over again. In the current version CMSImport will report what is wrong and allows you to fix those issues.

Now with documentation

The whole import process and API is documented now. You can download the manual from the CMSImport site.

Delete failed import documents that were created during import

When a new document is created during the import process and the import process fails the document will be deleted instead of left in the tree.

Improved performance by not setting null values

When a value contains null it will not be mapped against the Document property. This will save a few database calls according to this blogpost. This is only for new imported  documents. When a document is being updated the null value could be the new value and therefore will be assigned to the property.

What’s next

The coming 1.2 version will include more FieldAdapters and support for third party Datatypes. This will be included in both the Free and Pro edition of CMSImport.. I hope to have this ready by the end of this year. I’m also working on version 2.0 that will support structured import out of the box. This will be released early next year. Structured Import will only be included in the PRO edition.

Get it while its hot

Download the free version from our.umbraco.org. Please vote up the package if you like it. Or buy a Pro license from the CMSImport website.

When I released CMSImport 1.0 two months ago, I briefly described the FieldAdapters feature that  I was going to build for version 1.1. Today I want to go more into detail about the FieldAdapters feature for CMSImport.

The problem

The reason I started to think about fieldadapters was because I wanted to solve a very common problem. As you might know Umbraco only accepts three kinds of  data; string, integer and datetime. What happens when you try to map a boolean value to  a yes/no Umbraco datatype? An exception is thrown.  The following screenshot contains several products that we want to import into Umbraco.

When we’ve mapped the InStore column to a yes/no datatype the import will fail because it can’t map the boolean value to a 0 or 1 and we will see the following ugly screen

And a FieldAdapter can fix that?

Yes a fieldadapter inspects the value and tries to parse it, in this case it will try to pares a true/false value to 1/0, best to explain by exploring the source code. The whole feature is based on the new IFieldAdapter interface.

          using System; namespace CMSImportLibrary.Interfaces
{ /// <summary>///
Implement the IFieldAdapter interface to convert an mallformed field to a correct
type./// </summary>publicinterface IFieldAdapter
{ /// <summary>///
Contains the GUID of the datatype we want to parse using this FieldAdapter///
</summary> Guid DataTypeId { get;
} /// <summary>///
Parse the data /// </summary>///
<param name="value">The value to parse</param>///
<returns></returns>object Parse(objectvalue);
} }

As you can see in the code snippet the interface contains the property DataTypeId. The value needs to correspondent to the Id (GUID) of the datatype you are creating the fieldadapter for. During the import a factory inspects the underlying datatype of the document property, if a fieldadapter is found it will execute  the Parse method. This will work on all Umbraco datatypes (also custom datatypes or datatypes from third party packages) as long as you know the Id of the datatype. You can find the id by opening the datatype in Umbraco, there you see the RenderControl .

When opening the source code of the rendercontrol you’ll see the Id of the datatype (a GUID). Don’t worry if you can’t open the source code of the datatype i’ll add the Id’s of all known datatypes to the documentation of CMSImport and I’ll make a free tool that shows the Id of any datatype.

Implementation of the BooleanFieldAdapter

Now enough with all the boring stuff, let’s see the fieldadapter in action. Below you’ll see the implementation of the BooleanFieldAdapter. I’ve mapped the id of the True/false datatype to the DataTypeId property so CMSImport knows that it needs to call the Parse method during the mapping of a yes/no document property. In the Parse method I simply check if the value is already in the correct format , if not it will try to convert the value to 1 or 0 and return that , otherwise it will just return the original value.

          using System; using CMSImportLibrary.Interfaces; namespace CMSImportLibrary.FieldAdapters.DefaultFieldAdapters
{ publicclass BooleanFieldAdapter
: IFieldAdapter { #region IFieldAdapter Members public Guid
DataTypeId { get { returnnew Guid("38b352c1-e9f8-4fd8-9324-9a2eab06d97a");
} } publicobject Parse(objectvalue)
{ if (!(value.Equals("0")
|| value.Equals("1")))
{ bool boolValue = false; if (bool.TryParse(value.ToString(), out boolValue))
{ return boolValue ? 1 : 0; } } returnvalue;
} #endregion } }

Now when we run the import  again it will just import the data without any errors, just by adding a few lines of code, isn’t that powerful?

Beyond fixing problems

Now that we have this mechanism we can also use it to modify data during the import. Let’s say we import a piece of content from an old site that contains an image tag. This will just run fine but when you have a reference to an image on your old site and  you delete that old site all the references to the images are dead. FieldAdapters can help solve this issue by inspecting the text, extract the image tags, import these images into the media library and update the image tag with a reference to the media item. This will really help keep your site consistent. The same applies for Upload fields.

Will all of this be included in the free version of CMSImport also?

All FieldAdapters that fix errors (Like the BooleanFieldAdapter) will be included into the free version of CMSImport. FieldAdapters that helps you updating the content will only be included in the PRO version of CMSImport. And again you can also create your own FieldAdapters, free for both versions.

This will be the major feature of the V1.1 release of CMSImport. Please let me know what you think about this feature.

I’m very pleased to announce that CMSImport 1.0.3 is released. Now I can hear you think CMSImport? Must be a fork of the great UmbImport package. No this isn’t the case. A few weeks back Niels (AKA @Umbraco) asked me to change the name since Umbraco HQ got a lot of requests about this package. So now the name is CMSImport and that’s not going to change anymore.

CMSImport PRO

Finally we’ve finished our commercial edition of CMSImport.  CMSImport PRO gives you all the options of the default package and the following extra features:

• Update Content
• Save Import Steps
• Schedule imports for a certain time and day

Pricing

You can buy a single domain license of CMSImport. With a single domain license you are allowed to use  CMSImport PRO for a single domain and all subdomains, such as www.example.com, accept.example.com, and local.example.com.

We also have a Enterprise license available. With an Enterprise license you are allowed to install the CMSImport PRO on unlimited production web servers, and use it for unlimited Umbraco instances within the Enterprise. 

A single domain license will be available for  99 Euro, an enterpise license for 389 euro.

When you buy a license you’ll get free updates within 90 days of purchase and  free updates for all minor releases within a major release.  For example, if you purchased a  1.0 version of CMSImport, you get free updates of all 1.x versions through our client area.

Special 1.0 offer.When you buy the 1.0 release you’ll get a free update to 2.x. This is a 1.0 offer only!

What’s more in this release?

Several issues are solved in this release(both in the free and Pro release):

• "item with the same key already added" error when using duplicate column names
• Automapping column names
• The imported document creator is not always the administrator anymore. It's using the logged in user now. When you schedule an import you can select the user that should be used as the creator of the document
• Special characters in CSV are now supported, we’ve changed the reader from ANSI text to Unicode
• Sometimes CSV replaced spaces with empty strings, this is solved now
• With member import you can now merge any member property into the template. Simply surround the member property with [#(property here)]
• Using a renamed Umbraco folder. This is possible now, although it will be better to change it after install, otherwise you have to install manually.
• We’ve removed the limitation to allow only one DataAdapter. We are thinking to build a DataAdapter pack which contains adapters to import from wordpress, Rss, Outlook, excel etc. These adapters will be available for free in the Commercial Edition and for a small fee for the Free edition.

Roadmap

In the 1.x version we will add the following functionality:

• FieldAdapters. Sounds boring but this is a big thing. When you import data now, sometimes the import will fail. For example if you import boolean as text (true/false) and want to store that in a True/False field in Umbraco it will fail. Umbraco expects that the value will be 0/1. FieldAdapters will solve this problem. If a insert of data fails. CMSImport will check if 1 o more FieldAdapters are available to convert the data in the right format. This will be added to version 1.1 which must be ready before CodeGarden 2010.
• Dictionary Import. Need I say more?
• Hierarchical imports(PRO only).

In the 2.x version we will add the following functionality:

• Hierarchical import support in Data Adapters. Not the same as the 1.x Hierarchicalimport feature ;-)
• Export/import definitions (PRO only). An easy way to deploy Import definitions

More Info

For more info, download, or purchase you’ll go to http://www.cmsimport.com/