Convert a Directory of Markdown Text Files for Dynamic XML Datasource Use
This is an open discussion with 10 replies, filed under General.
Search
I’m still weighing up a local storage for all my stuff (Yojimbo, Yep!, DevonThink et al) and yet to find the perfect solution. I am, however, looking to store more than text files.
But this is a neat idea. I’d suggest adding some more meta data to your XML so you can sort and filter. For example:
<?php // ensure output is sent as XML header('Content-Type: text/xml'); echo('<?xml version="1.0" encoding="utf-8" ?>'); include_once('markdown.php'); $handle = opendir('.'); echo('<files>'); while (false !== ($file = readdir($handle))){ $extension = strtolower(substr(strrchr($file, '.'), 1)); // only process markdown text files if(!in_array($extension, array('txt', 'md', 'markdown'))) continue; $text = file_get_contents($file); echo('<file name="' . $file . '" last-modified="' . date("Y-m-dTH:i:s", filemtime($file)) . '">' . Markdown($text) . '</file>'); } echo('</files>'); ?>
Attach this script as a Dynamic XML data source and you get the file list including file names and dates. Using XSLT you could choose to display a list of file names only, and sort them by this date. The following assumes you have a Symphony page “Markdown Files” that accepts a URL Parameter named filename
:
<?xml version="1.0" encoding="UTF-8" ?> <xsl:stylesheet version="1.0" xmlns:xsl="http://www.w3.org/1999/XSL/Transform"> <xsl:output encoding="UTF-8" indent="yes" method="html" /> <xsl:param name="filename"/> <xsl:template match="/"> <xsl:choose> <xsl:when test="$filename=''"> <ul> <xsl:apply-templates select="//file" mode="list"> <xsl:sort select="translate(@last-modified,'-T:','')" data-type="number" order="descending"/> </xsl:apply-templates> </ul> </xsl:when> <xsl:otherwise> <xsl:apply-templates select="//file[@filename=$filename]" mode="render"/> </xsl:otherwise> </xsl:choose> </xsl:template> <xsl:template match="file" mode="filename-list"> <li> <a href="/markdown-files/{@filename}/"><xsl:value-of select="@filename"/></a> </li> </xsl:template> <xsl:template match="file" mode="render"> <h1><xsl:value-of select="@filename"/></h1> <xsl:copy-of select="*"/> </xsl:template> </xsl:stylesheet>
It should first list the files if the $filename
URL Parameter is not set. The list of files will be sorted newest first, using xsl:sort
. The translate()
function should remove any non-numeric characters from the date string, leaving a number that can be sorted on. If the URL parameter is set, then the full file is shown.
This is how I’d approach it. Keep the data source as simple as possible by just providing the least amount of data required, but perform the manipulation and rendering of the data using the XSLT layer.
Wow, thank you very much, Nick!
I will find some time to experiment with this.
Much appreciated.
Regards,
Rick
I took @nickdunn's script above and combined it with a script that I found on Stack Overflow, that will allow you to recursively search for all Markdown files and then list and convert them into one XML document...
I have it on this GIST here.
Also, I made an attempt at updating the script so it would also work with YAML Front Matter in Markdown files and output the YAML Front Matter under a <meta>
element.
I did this with my rudimentary PHP skills, so if folks can make it better, please do. ALSO, right now in the YAML Front Matter output in the XML... I have it static outputting a "title" and "uri". Since it will accept whatever you give it, I am trying to figure out how to dynamically produce the XML nodes for meta.
So, for Symphony purposes, if you have a few directories full of markdown files (with YAML Front Matter) that you want to make into a XML data source, you can do with this script.
I have created script to recursively read through a "content" directory of Markdown files that contain YAML Frontmatter (ala Jekyll or Statamic) and would like to create an XML document from it to use as a dynamic datasource.
PHP HELP NEEDED
I was wondering if you could look at my script, located here, https://github.com/bzerangue/markdownyamlxml/blob/master/index.php#L131-L138
I was wanting to dynamically create xml nodes based upon whatever YAML key:value pairs are available to me. Is there a way for my dynamically create the fetch statements to fetch me whatever key:value pairs are there for me to use
Unfortunately, my PHP skills are completely horrible and would love to learn more. If and only if you have time, would you mind looking at that script and giving me some pointers?
Brian, is there an example YAML file in the repo?
EDIT: ignore me! RTFR (Read the bloomin Repo) :)
Maybe Dayle Rees' repo for his static laravel based website document parser may help inspire some coding convention: Especially the parseMetadata function: github.com/daylerees/kurenai
@moonoo2 - Thanks for the response. Actually, Blaxus, the author of the PHP class for handling YAML frontmatter in PHP, helped me with how to get it to do what I wanted it to. I've updated my little example repo. This can be used to create XML for a directory full of markdown files with YAML Frontmatter.
Brian, Try using the fetchKeys()
function mentioned here: https://github.com/Blaxus/YAML-FrontMatter/blob/master/frontmatter.php#L45
So take your $page variable and grab all meta keys as an object: $metadata = $page->fetchKeys()
Then loop over the $metadata
array to grab the key value pairs.
Hah.. I'm too slow.
I see you made the $data public instead of private so that would work as well, here's what I did without altering the FrontMatter class:
$metadata = $page->fetchkeys(); foreach($metadata as $key => $value) { # You want to skip the content item right? # $key = title # $value = $page->fetch('title') //var_dump($key); $metatitle = $doc->createElement(trim($key)); $metatitle->appendChild( $doc->createTextNode($value)); $meta->appendChild($metatitle); }
@moonoo2 - thanks for the input that helped a lot.
Create an account or sign in to comment.
I had been looking into a way to store and display for my own purposes a set of topical text files, and had been eschewing somewhat-heavy local database “shoebox” type information managers such as DevonThink or Yojimbo on the Mac. A simple text file is enough for many purposes, so I initially tried Notational Velocity, because of its capability to sync to SimpleNote. Further, NV can be set to allow one to access the data folder (
~/Library/Application Support/Notational Data
) directly, so that markdown-formatted text files can be saved there with a regular text editor such as TextMate or SubEthaEdit, for a clean and easy workflow.However, although I can indeed view SimpleNote on iPhone or via a web browser, I am uncomfortable with putting certain stuff in the cloud, and so would rather self-host so I have control over storage, backup and dissemination. I experimented with using Gollum, a ruby Git-based wiki, but it’s served via Sinatra, and so not so easy to secure out of the box with something like an
.htaccess
file. There are plenty of other static site generators as well, such as Jekyll or nanoc, which I am sure would also work fine.But why not try it in Symphony CMS? Why not indeed. Nick Dunn generously provided the hints to get me started, and I was able to cobble together a (quite clumsy) PHP script that parses markdown files placed in a web folder, and presents the same in an XML file that can then be used as a Dynamic XML Data Source for Symphony CMS pages. I am not a coder, obviously, but this is the script:
Triers will need to get the markdown.php library, and for better or worse, I just chucked that, the above php script, and a few markdown files, into the root of an exposed site folder. When I run the php file, it produces visible results in the browser, as well as producing visible results in the
?debug
view for the Symphony CMS page it is attached to. That lets me do a few different things:A few challenges and ideas and questions come to mind already:
Any and all comments are most welcome and appreciated. Perhaps I can somehow improve this so it is useful to people who want to edit content in a text editor.
Sincerely,
Rick