Symphony.

This interesting thread sparked some conversation about the need for modular ensembles. I (like others I’m sure) have put alot of thought into this problem over the years. I started this thread to further explorer everyone’s ideas on the topic.

Here is an attempt at describing the gist of my approach:

Concepts for Modular Ensembles

Enabling ensembles to be added to existing Symphony installations has always been a bit of a ‘Holy Grail’ in my mind for Symphony. This is because if it were well-executed, it would elevate Symphony (as perhaps the first?)into a new generation of Semantic CMSs. Not just producing semantic content (which simple enough to do now if you need to) but actually using semantics to organize itself.

General Requirements

• Allow Page/Views and all related symphony items (DS, Events, Sections, XSLT) to be added to an existing Symphony site automatically, without interfering with the existing site.
• Allow selected Page/Views to similarly be exported from an existing site.
• Define the external 3rd party extensions required for these ensembles to be used
• When necessary, integrate with existing site data models

A semantic approach to modular ensembles

Unlike the current concept of an ensemble, a modular ensemble would need to be referential, rather than simply a snapshot of the site you wish to share. Such an ensemble needs to describe the Page/Views, datasources, events, and sections it uses. It also needs to include (probably as source, possibly as reference) the XSLT it uses. The most difficult aspect required for a complete solution would also be reference to the linked but un-included data members. For instance, a blog ensemble will probably want to use a Members data member, but rather than including one itself, defer to some other ‘member’ section installed on the system. In sum, a modular ensemble needs to be a definition document, and I see no reason this document shouldn’t utilize standards.

I think the best approach for this will be to model everything in standard semantics (RDF/S). Every field in Symphony should be classed by its more primitive datatype format. When it is added to a Section, each field should be sub-classed to describe how it relates to the section in which it is placed (e.g a textfield field might have an XSD definition as a string, but when added to a ‘Members’ section, it is classed as ‘First Name’). Essentially, this would mean turning the Section editor into a basic RDFS authoring interface. This is important, because it addresses issues of naming: A section, or field, that has a semantic class can be instantiated with whatever name you like. This allows us to maintain the normal experience of naming our sections and field whatever we’d like.

The need for XSLT namespacing

This would come at the cost of increased complexity in the transformation. Rather than the simplistic null-namespaced transformations Symphony utilizes now, every semantic section(ie any section touching the modular ensemble system) would need to be namespaced in its xml datasource output. This would be necessary for allowing the ensemble’s XSLT to target its data without needing to know the specifics of its instance. For example, if I have “Journal” ensemble and a “Blog” ensemble, both of which have an “Articles” section, the XSLT fot both would target the “Articles” section. No good. Rather, the XSLT will need to target “blog:articles” or “journal:articles”. Keeping this as simple as possible would be very important, since it is generally the least-known aspect of working with XSLT both among the Symphony community and XSLT users in general, I’d imagine.

Data model linking

One powerful advantage to this semantic approach would be the ability for ensembles to ‘know’ where they should fit into the data model of the site they are being installed into. Referring to my blog example, If a site has a Members ensemble installed, then it will have a semantically defined ‘Members’ section available. When an ensemble requiring members is installed, it could search for a Members class in the existing site datamodel. Upon finding it, it could either automatically or through a user prompt, link to the existing section.

These are just some basic ideas related to this solution. There is much more that would need to be fleshed out to determine the validity of this approach. I do feel strongly that if Symphony were to accomplish this using standards-based semantics, it could be a radical step toward making Symphony a next generation CMS.

The above is either way over my head or it is gibberish — probably the former. IMHO, an ensemble is simply an offering by someone that may or may not be useful to others. The author is simply saying: “This is how I satisfied a requirement that I had. You may use any part or all of it as you see fit.”

I suspect that if a lot of structure is imposed on ensemble developers we will see fewer people developing and publishing ensembles. I would like to see a greater variety of ensembles published. Some might emphasize design more than content and its handling and others might emphasize content and its handling more than design. A designer might take an ensemble put together by a techie, improve its design and republish it, or vice versa.

Look at this as just a voice from the sidelines. I’m not a professional; just a hobbyist. But be careful that you don’t over-complicate this.

The above is either way over my head or it is gibberish — probably the former

Actually, probably somewhere in between ;). Simply put, I’m suggesting using RDFS as the engine for allowing portable structures in Symphony (call them ensembles or whatever).

an ensemble is simply an offering by someone that may or may not be useful to others.

Right, this is their current function. The other thread I linked to includes some discussion of needing something that is more functional than demo ensembles, but more generalized than an offering as you describe. This is where the topic of modularity came up.

I suspect that if a lot of structure is imposed on ensemble developers we will see fewer people developing and publishing ensembles.

Yes, this is critical. The ideal implementation of creating RDF schemas would probably be done entirely within 2 locations in the admin interface of Symphony: in the Section editor screen, assigning RDF classes to fields, and in an ensemble creation screen, selecting the Page/Views to be added to the ensemble.

Like I mentioned, the bigger issue would be namespacing in the XSLT, since this would probably be necessary to some extent. That could make this approach a non-starter, or at least less approachable.

Also, keep in mind that this would probably be a separate beast from an Ensemble. Outputting a structure-only, installable ‘image’ of a Symphony site(the current ‘Ensemble’) would not necessarily intersect this functionality.

Some of my work with ensembles attempts to limit the complexity of trying to build a single “super ensemble” by breaking it down into manageable pieces to figure out how each one could work. The challenge has always been, how do I then assemble the pieces into a coherent whole, especially when the configuration is tied to database entries and their relationships to section IDs. It has meant manually rebuilding sections for each install.

Symphony 3 Section Configuration

The approach of Symphony 3 is to remove the configuration of sections from the database and instead store the settings in the form of an XML document. For example, the following XML file stores the settings for the configuration of a Categories section with two fields: Title and Description.

<?xml version="1.0" encoding="UTF-8"?>
<section guid="4c2f3e2cf22a9">
  <name handle="categories">Categories</name>
  <hidden-from-publish-menu>no</hidden-from-publish-menu>
  <navigation-group>Content</navigation-group>
  <publish-order-handle></publish-order-handle>
  <publish-order-direction></publish-order-direction>
  <fields>
    <field guid="4c2f3e2cf2300">
      <required>no</required>
      <show-column>yes</show-column>
      <size>medium</size>
      <type>textbox</type>
      <element-name>title</element-name>
      <name>Title</name>
      <publish-label></publish-label>
      <text-size>single</text-size>
      <text-formatter>markdown_extra_with_smartypants</text-formatter>
      <text-validator></text-validator>
      <text-length>none</text-length>
      <column-length>75</column-length>
      <text-handle>yes</text-handle>
      <section>categories</section>
    </field>
    <field guid="4c2f3e2cf23cb">
      <required>no</required>
      <show-column>yes</show-column>
      <size>medium</size>
      <type>textbox</type>
      <element-name>description</element-name>
      <name>Description</name>
      <publish-label></publish-label>
      <text-size>small</text-size>
      <text-formatter>markdown_extra_with_smartypants</text-formatter>
      <text-validator></text-validator>
      <text-length>none</text-length>
      <column-length>75</column-length>
      <text-handle>yes</text-handle>
      <section>categories</section>
    </field>
  </fields>
  <layout>
    <column>
      <size>large</size>
      <fieldset>
        <name></name>
        <field>title</field>
        <field>description</field>
      </fieldset>
    </column>
    <column>
      <size>small</size>
      <fieldset>
        <name></name>
      </fieldset>
    </column>
  </layout>
</section>

This will allow the exchange of configuration files between Symphony installs and enable the creation of sections and fields by simply loading an XML file. Creating modular ensembles becomes far easier this way. Plus, all configuration can be in source control and exist apart from entry data existing in the database.

So, how would you see RDF being used in Symphony 3?

I think it makes sense to indicate a datatype for each field: string, boolean, integer, etc.

Are there other schemas that are worth considering?

• Adobe’s XMP
• PRISM

I think it makes sense to indicate a datatype for each field: string, boolean, integer, etc.

When you say “field” do you mean a Symphony “Field” or an elemental database field (column) in a table? If the former, then don’t forget that fields are not always clear cut string, integer etc. Symphony’s “data types” are far broader than that.

My knowledge of RDF and XSD is mediocre at best, so all of this is beyond my understanding too.

Modular Sections

Typically, a module or feature includes a group of sections, events, data sources and extensions that are meant to work together, perhaps you could have a Module XML file that contains the RDF data with a description of the feature, as well as information about the dependencies and relationships between key sections and fields.

That way, complexity is added as a completely optional layer on top of the concepts that Symphony users must understand to use the system. If modules could interact with something like alpacaaa’s Builder ensemble, it might be possible to activate and deactivate modules as suites of sections, events, data sources, extensions, pages and utilities.

@nickdunn, I mean the data type of the database field.

I was thinking of how I might use Form Controls and the schemas described by the section configuration files to be able to emulate in front end forms something similar to what Formtastic is used for in Ruby on Rails applications. Classes are inserted based on the data type of the form field.

Clients are asking for the ability to assemble their own forms. I’m trying to think ahead about how I would go about providing this functionality.

Plus, I could provide another use case for Symphony in our agency to easily create prototypes with forms and validation and other reusable content modules with flexibility similar, or perhaps far beyond what Formtastic is able to achieve.

By way of disclaimer, I am no RDF wizard myself, but I have been studying it off an on (in part to address this issue).

So, how would you see RDF being used in Symphony 3?

For starters, you could define RDF classes for sections and fields based on this XML. That might not make any immediate functional difference (though it would be very forward-thinking in utilizing standards over idiosycratic XML).

If RDF were used for defining Modules, creating a section would not only create an instance as the XML above does, but also create a class to be machine readable elsewhere. Simply duplicating and transplanting the above XML might do the trick to modularize sections, but there would still need to be a parsing layer to handle name collisions and other instance- and context-related issues.

These issues would center around the differences in instance data: linked fields, element names, etc. Even things like column-length have nothing to do with the inherent semantic structure you’re trying to transport. Having to create some system to handle all this, specific to Symphony’s XML definition of a Section, seems, dare I say, un-Symphony, when there are such widely-used standards available meant to address very similar problems.

And the potential advantages are enticing: Consider a “Blog” section class that has as a requirement (as defined in its RDFS) of a “Date” field class. Field extension developers could create fields that satisfy (and go beyond) the needs of the date field class, extending it. Module consumers could choose from any “Date” field (that is a subclass of the Date field) without breaking their datasources, or perhaps even the XSLT related to the module.

@nickdunn

The issue of datatype is tricky. Any data in Symphony has 2 facets: the input (and it’s required interface) and the data view (which can be fairly specific XML). Having to define that in a standards-compliant way might be overkill (imagine having to write an XSD when authoring a field extension because you are outputting custom XML to the datasource).

Extension dependency could be a real issue. The best would be to have a URI for every major version of every extension, which could be difficult. Even if we had that, describing things like which page/views get an extension’s events could be problematic.

It maybe because I’m totally tired or because I don’t know much about RDF but while I like the idea of modular ensembles the described concept seems to convert Symphony’s beauty in simplicity into a bureaucratic monster.

Andrew, is it as complicated as it sounds? And does it need to be that complicated?

@Nils

No, it does not need to be as complicated as I’m describing it. Attempting some of the features I’ve mentioned would have to be the product of some serious evolution and iteration.

A few questions in my mind:

1. Is Symphony 2 or 3 the best target for developing a Module capability?
2. Should a module system maintain awareness of it’s modules, or should it simply automate the process of migrating pieces of a Symphony site from one install to another (not requiring any sort of persistent module management or tracking)?

I really like the idea to have bigger building blocks in Symphony, like whole modules as you describe, Andrew, or like a subset of a section (just a set of fields) you can include in different sections which are not a section on its own and not a subsection, more like a prototype for fields you share between multiple sections. I talked with Brendan and Rowan about the latter, and this could be done as an extension in Symphony 2. I reckon that the former is better suited to Symphony 3, mainly because the structure of a section is saved in the filesystem.

I haven’t thought so much about an implementation, only started thinking conceptually about it when I first saw Symphony 3. And I first (until your RDF elaboration) thought only about having a set of predefined sections, just for a head-start. But the whole RDF thing seems to make sense :-), just started to read about RDFa. Have to wrap my head around it and let the topic sink a bit.

The Setup feature in the Members extension (1.2.1beta) is an example of how extensions can be used to automate the process of creating a section. I was pretty excited when I saw this, since it is a working proof-of-concept that could be extended to create modules in Symphony 2.

Ok, I have an update:

I’m assuming sections are the place to start (all other ensemble components build from sections, and there is no need to deal with the issue of how to handle the param pool). Even so, there is a basic problem: there is no way to know what section field configuration values (i.e. columns in the sym_fields_*type* table) are inherent to the definition of the field instance (which we want to transfer) and which are contextual, and should be configured at the receiving end (usually IDs of other sections etc.).

The only way to address this would be for someone (ideally the field author) to define this somehow. I see field values falling into a few classes:

1. Inherent value; to be passed through to the imported section/field
2. System context value; to be reassigned by the importing system (the section and field IDs)
3. User context value; to be reassigned by the importing user (section links, dynamic select links, subsections, etc)

The user context values are important, because they will need some type of interface for the importing user, even if its just a list saying “These values are probably wrong, and need to be reset.”

While some values must be reassigned by the user for the section to function, others could optionally be reassigned. For instance, the section name itself, or really the title of any of the section’s fields could potentially be set by the importing user.

I was originally intending to just pass all values through, and let the importing user sort them out. This might work for basic testing, but it really won’t do for practical use. We’ll have to address this sort of complexity eventually when it comes to data sources: how do you handle param pool variables used in a DS you want to export ?

Any thoughts?

Hi all,

Getting into this again, and I still have not resolved the issue of which field data to export, and which to reset. What would field extension developers think if they could define a public function similar to About() that would define which of their fields’ DB field values should travel with a field on export, and which should be cleared and redefined at the destination Section?

Some uniform way to define this is needed for even basic Section transport, but it will be even more important when I add more complexity, specifically an “Include related Sections” option.

For right now I’m going to define the ‘baseline’ DB fields that should always be exported, but that is going to diminish the utility of the exporter.