Symphony.

After a long break from developing with Symphony (although still being around) I've come back to an old thorn that really bugged me, and it turns out, still does.

A loonngg time ago, I discussed the Reflection Field not working with any auto-uploader/importer or anything that didn't save the entry by the entry save button, on the edit page.

This is still the case.

I remember @brendo mentioning a way it could be updated to work with these automagic fields, but I can't remember what it was.

Does anyone have the inclination to fix the reflection field?

Rowan is the active maintainer, and added updates for 2.2 compatibility and fixed other bugs in March. I presume Rowan doesn't have the time or inclination to bug hunt if he's not actively using the field in current projects. So if you hit a bug, be sure to post it up in the usual way and we'll figure it out.

At a quick glance this is because the field relies on delegates (EntryPostCreate, EntryPostEdit) to be able to fully build the entry XML after it has saved, and then run XPath against it. Because these delegates are in the UI (the code that builds the backend entry forms) and not the EntryManager class itself, they will only execute when you save the entry through the backend. This will be the case for many other fields too (such as Entry Revisions).

I'm not sure the field itself can be modified to work without this, but certainly these other extensions that do the saving should be updated to properly fire the respective delegates. For example Brendan recently updated Union Data Source to implement the DataSourceEntriesBuilt delegate. Implementing these delegates is a case of copying the notifyMembers method call and passing through the correct parameters.

Which extensions would need to be updated?

We discussed that in Symphony 3 delegates should be split between the UI and the API. Things like entry saving should happen at the class level, so any code calling $entry->save() would fire the save delegates, rather than the delegate execution being attached to the controller code for the backend page. But that's by-the-by, we're in Symphony 2 right now.

Unless, of course, Brendan had a remarkably cunning plan, so cunning that it would out-fox a cunning fox who had just graduated from the Oxford University of Cunning with a PhD in Cunning, to make it work without the delegates...

Unfortunately, the cunning plan will have to wait...

Anyhoo, yeah so I did a bit of hunting from the past and found out that Brenden had added loads of delegates to the XML Importer that fire after the entries have been made, so I suppose that I could modify the Reflection field to pick up on these delegates too?

Also the Mass Upload Utility is one to look at too, I think it works in the same way as the XML Importer... It may need some delegates too?

The delegates for the Mass Upload Utility should be automatically fired now since I don't duplicate any of Symphony's core; it just uses an event now thanks to Nick, so all delegates should be working!

Actually, I'll take that back, I haven't tried it since all your updates, apologies.

Does it work with the Reflection field now? Or will the reflection field need updating to use the delegates better?

If XML Importer fires the delegates, the Reflection field should not need any updates. And because Scott has updated Mass Upload Utility to use a Symphony event as opposed to using the EntryManager class, the event will handle the correct delegates natively.

The other extension I can think of is Giel's Import/Export CSV which should probably implement the delegates too.

Any others that do entry importing?

I can't think of any without a good look through...

So, is it the XML Importer that needs updating, or the Reflection Field that needs to catch the XML Importer's delegates. I still have trouble getting this delegate stuff...

The Reflection Field needs updating to listen to the XMLImporter's delegates.

Think of delegates as a conversation. One extension says, 'I'm done with, 'x', last call anyone?' and everyone who has something to do can listen for 'x' and then jump in with whatever logic they need to do.

Some delegates are just like that, they just announce things have happened, eg. FrontendInitialised, others provide parameters by reference so you can impact where the rest of the 'conversation' is going to go, eg. FrontendPageResolved.

Why doesn't the XML Importer implement the same delegates as the publish page: EntryPostCreate, EntryPostEdit etc? Otherwise it means that all field extensions that rely on delegates will have to subscribe to the delegates that have similar names for other extensions.

To be honest, I don't remember the full reason.

I think at the time it was so you could do something different if an entry was imported and not created via the backend. This can be achieved anyway by comparing the '$context' of the delegate, where the standard delegate is '/publish/edit/' and the XMLImporter is '/xmlimporter/importers/run/', so it's a bit of a moot point.

I'll update it when I get to the office, and post results here.

Thanks for all your help guys!

Ok, so I added the delegates into the Reflection Field, and it fouled up.

I also had to add the Subsection Manager into the OR list of fields that allow multiple select, which works fine.

Running the importer with my SSM change runs instantly and imports all my entries, obviously without populating the Reflection Field.

With the delegates in the Reflection Field, and reenabled to add them to the DB, the importer runs out of memory after about 30 odd entries, at 64m allowance in PHP with the following error message:

Fatal error: Allowed memory size of 67108864 bytes exhausted (tried to allocate 46 bytes) in /Users/johnporter/Projects/Websites/Open Book Publishers/assets.openbookpublishers.com/httpdocs/symphony/lib/toolkit/class.fieldmanager.php on line 198
Call Stack: 0.0002 341192 1. {main}() /Users/johnporter/Projects/Websites/Open Book Publishers/assets.openbookpublishers.com/httpdocs/index.php:0 0.0224 2240196 2.
Administration->display() /Users/johnporter/Projects/Websites/Open Book Publishers/assets.openbookpublishers.com/httpdocs/index.php:25 0.0224 2240840 3.
Administration->__buildPage() /Users/johnporter/Projects/Websites/Open Book Publishers/assets.openbookpublishers.com/httpdocs/symphony/lib/core/class.administration.php:375 0.0450 4427484 4.
contentExtensionXmlImporterImporters->build() /Users/johnporter/Projects/Websites/Open Book Publishers/assets.openbookpublishers.com/httpdocs/symphony/lib/core/class.administration.php:222 0.0450 4427484 5.
contentExtensionXmlImporterImporters->__prepareRun() /Users/johnporter/Projects/Websites/Open Book Publishers/assets.openbookpublishers.com/httpdocs/extensions/xmlimporter/content/content.importers.php:41 0.5566 7027016 6.
XMLImporter->commit() /Users/johnporter/Projects/Websites/Open Book Publishers/assets.openbookpublishers.com/httpdocs/extensions/xmlimporter/content/content.importers.php:74 63.8492 63344716 7.
ExtensionManager->notifyMembers() /Users/johnporter/Projects/Websites/Open Book Publishers/assets.openbookpublishers.com/httpdocs/extensions/xmlimporter/lib/class.xmlimporter.php:380 63.8494 63345576 8.
Extension_ReflectionField->compileBackendFields() /Users/johnporter/Projects/Websites/Open Book Publishers/assets.openbookpublishers.com/httpdocs/symphony/lib/toolkit/class.extensionmanager.php:566 67.0305 65590328 9.
FieldReflection->compile() /Users/johnporter/Projects/Websites/Open Book Publishers/assets.openbookpublishers.com/httpdocs/extensions/reflectionfield/extension.driver.php:141 67.0305 65590328 10.
Extension_ReflectionField->getXPath() /Users/johnporter/Projects/Websites/Open Book Publishers/assets.openbookpublishers.com/httpdocs/extensions/reflectionfield/fields/field.reflection.php:263 67.0379 65623692 11.
fieldSubsectionmanager->appendFormattedElement() /Users/johnporter/Projects/Websites/Open Book Publishers/assets.openbookpublishers.com/httpdocs/extensions/reflectionfield/extension.driver.php:112 67.0473 65720932 12.
FieldManager->fetch() /Users/johnporter/Projects/Websites/Open Book Publishers/assets.openbookpublishers.com/httpdocs/extensions/subsectionmanager/fields/field.subsectionmanager.php:736

There's no formatting on the message either, just a dump to the screen.

The 3o ish entries that are created, also don't have the Reflection Field populated :o(

Any ideas anyone on how this can be fixed?

Just a quick note, if I do the above without the SSM changes, it is exactly the same problem, without my SSM entries having more than one entry. So I don't think it's an SSM problem here.

So, I've spent the day trying to understand how this all works. I've pulled the data apart that is sent to the Reflection Field, followed it all the way through the Reflection Fields compile function, which always returns true on the database update statement at the bottom of the function, which says to me that the data is added to the database (is it?).

It is not. I can't figure out for the life of me where this data is going to and why it isn't written to the database?

I've even added a little check into the bottom of the function to see whether a SELECT statement on the entry_id will return true, which it does, even though there is no data in the table ??!!??

I've got to the end of where I can with this. The issue is definitely the Reflection field, as it happens with both the XML Importer and the Mass Upload Utility, both latest versions.

EDIT: Could it be something to do with the fact the checkPostFieldData function isn't called when not on the section entry edit page?

Sorry for my ignorance, but what exactly is the problem? Is it that the Reflection field value is never set when importing via the XML importer? So for example if you had a section with three fields: "Title", "Author" and "Concatenated" (the latter being a Reflection field concatenating the former together), when importing via XML Importer this field would never be given a value?

That's correct, and with the Mass Upload Utility, no ignorance there at all!

It's definitely down to how the Reflection Field does it's magic, and I think it's down to the checkPostFieldData function and how it sets values back to the driver for the driver to answer the delegate callbacks.

I would just need to figure out how to modify the extension to recreate this action when it's not being used on an edit entry page. I just don't know where to get the field from.

Well, I'm a little miffed at myself for not thinking the obvious here.

If I attach the Reflection Field to the Importer, and pass any value to it (like 0), it fills in the details.

That is not at all like how I expected it to work.

It did require the XML Importer's delegates to be registered with the extension, so I will do a pull request on that ASAP.

I can probably shed a little light here. XMLImporter, from what I remember, works the same way as an event. If the field is not passed, it'll never be called, so it'll never update.

I imagine this is the case with the Reflection field, and you could probably test it by just doing a simple event that excludes the Reflection field. It shouldn't work, as it's never processed.

By you including the Reflection Field in the XMLImporter, it has to process.

I see. (I think).

It still isn't populated by either the CSV Export/Import extension or the Mass Upload Utility, but I haven't got the time to figure out why.

Those extensions probably don't provide the necessary delegates :)