Symphony.

This discussion started in the XML Importer thread:

me:

I'm in the need of an JSON importer which is actually doing exactly what this extensions does. The only difference is that instead of an XML file/feed I'd like use a JSON response of an external API as source.

What's the best idea?

• Creating a new extension, JSON Importer, which is more or less a copy of this extension.
• Forking and extending XML Importer allowing JSON sources.

What do you think?

phoque:

You could also create a simple JSON-to-XML PHP script and feed its result to the importer.

me:

What's the advantage beside talking XML? If I have an external API that returns a - normally documented – JSON object why should I translate it first when I can import it directly? As it's about importing data into the database and not about templating I don't see any advantages in using XML as a middleman.

The only difference between a JSON fork and the existing XML Importer will be the way the values are resolved, but almost everything else will be the same:

• supply a URL that returns a JSON string
• match up elements in the feed (JSON) to your section fields
• run the import, create new entries or update existing ones

To this end, I suggest either a fork, or updating the XML Importer to make it more generic so that it can be supplied XML or JSON. The logic that grabs a feed, the UI to match up feeds with section fields, and the logic to create/update entries should be re-usable so it would be a waste of your time to write this from scratch.

If I have an external API that returns a - normally documented – JSON object why should I translate it first when I can import it directly? As it's about importing data into the database and not about templating I don't see any advantages in using XML as a middleman

The biggest benefit would be time. If we've already got an XML Importer and it will only take you half an hour to implement a conversion script from JSON to XML, then this is easier than spending two days rewriting the whole thing to use JSON natively. How critical is importing JSON? ;-)

As it's about importing data into the database and not about templating I don't see any advantages in using XML as a middleman.

It would be dead easy. No new extension required. :-D

Gotta say:

1. I would love native JSON import functionality, and wish I had the time to help build this. I'll be glad to thoroughly test anything you guys come up with.
2. I'm already doing JSON to XML translation quite a bit, and it hasn't been working very well for me in general. It's possible that the libraries I'm using for conversion are simply mediocre (I've tried several), but I think there are quite a few quirks and issues that arise from doing this on generic/unknown datasets — performance penalty for translation being one, as well.

That said, built-in translation would be a substantial improvement over my current situation, so I'm not going to pick any nits here! :)

For a project I worked on earlier last year I customised the XMLImporter to work with JSON (I read Facebook's API). I'll try to find some time to clean it up and post it.

Brendan, have you found some time to look at your JSON importer or would it be possible that you provide it as is?

Haven't the time, so I've just posted it relatively as is on gist.

Slight background; we use a CRON job to hit this URL every so often which will pull in the albums from Facebook, and then import them into the system using a XML Importer which is created programmatically instead of via the backend's 'Run'

I've removed our locking processes (using Tasklock extension) which prevents the server from going down should a previous CRON fail to complete.

Thanks!