Symphony.

Ok, so I'm still having major issues with doing imports and the reflection field. I have set my servers memory for PHP at 128mb and it is still topping out and failing when trying to import 123 entries. It fails at 55 entries now.

Can anyone shed some light onto this?

Fatal error: Allowed memory size of 134217728 bytes exhausted (tried to allocate 69 bytes) in /Users/johnporter/Projects/Websites/Open Book Publishers/assets.openbookpublishers.com/httpdocs/extensions/subsectionmanager/fields/field.subsectionmanager.php on line 710 Call Stack: 0.0006 339036 1. {main}() /Users/johnporter/Projects/Websites/Open Book Publishers/assets.openbookpublishers.com/httpdocs/index.php:0 0.0281 2239756 2. Administration->display() /Users/johnporter/Projects/Websites/Open Book Publishers/assets.openbookpublishers.com/httpdocs/index.php:25 0.0281 2240400 3. Administration->__buildPage() /Users/johnporter/Projects/Websites/Open Book Publishers/assets.openbookpublishers.com/httpdocs/symphony/lib/core/class.administration.php:375 0.0501 4477636 4. contentExtensionXmlImporterImporters->build() /Users/johnporter/Projects/Websites/Open Book Publishers/assets.openbookpublishers.com/httpdocs/symphony/lib/core/class.administration.php:222 0.0501 4477636 5. contentExtensionXmlImporterImporters->__prepareRun() /Users/johnporter/Projects/Websites/Open Book Publishers/assets.openbookpublishers.com/httpdocs/extensions/xmlimporter/content/content.importers.php:41 0.5637 7137476 6. XMLImporter->commit() /Users/johnporter/Projects/Websites/Open Book Publishers/assets.openbookpublishers.com/httpdocs/extensions/xmlimporter/content/content.importers.php:74 152.1208 130987296 7. ExtensionManager->notifyMembers() /Users/johnporter/Projects/Websites/Open Book Publishers/assets.openbookpublishers.com/httpdocs/extensions/xmlimporter/lib/class.xmlimporter.php:380 152.1210 130988188 8. Extension_ReflectionField->compileBackendFields() /Users/johnporter/Projects/Websites/Open Book Publishers/assets.openbookpublishers.com/httpdocs/symphony/lib/toolkit/class.extensionmanager.php:566 152.4840 131264580 9. FieldReflection->compile() /Users/johnporter/Projects/Websites/Open Book Publishers/assets.openbookpublishers.com/httpdocs/extensions/reflectionfield/extension.driver.php:141 152.4841 131264580 10. Extension_ReflectionField->getXPath() /Users/johnporter/Projects/Websites/Open Book Publishers/assets.openbookpublishers.com/httpdocs/extensions/reflectionfield/fields/field.reflection.php:263 152.4912 131290408 11. fieldSubsectionmanager->appendFormattedElement() /Users/johnporter/Projects/Websites/Open Book Publishers/assets.openbookpublishers.com/httpdocs/extensions/reflectionfield/extension.driver.php:112

Is there no memory clearing during the process after each entry? Because of the processes that each extension goes through (XML Importer running xPath, Reflection Field running xPath, Subsection Manager running xPath) the memory and objects that are being thrown around are getting huge.

It's a good job I can do this locally and not have to worry about bulk importing on a staging or production server, I currently have it running at 512mb memory for PHP and 300 seconds timeout... Which has worked.

IMO there is a problem here that needs addressing. I am trying so hard to sell this as a viable system to my employers for handling data, as we're taking on bigger 'data-centric' clients, but this could be a massive spanner in the works if every time I try and import to a section with a reflection field, it breaks.

I just assumed that by using an Event, it would trigger the EntryPre/PostCreate event, but having a quick look, I don't see it. Is this the event you're trying to get to trigger?

If so, nick will have to update the rest API extension too I suppose; I didn't see it in there either.

Hi Scott..

The delegates in the Reflection Field are: EntryPostCreate and EntryPostEdit and EntryPostSaveFilter

And I have included: XMLImporterEntryPostCreate and XMLImporterEntryPostEdit

They fire fine now for XML Importer, but not your extension, or Import/Export CSV either.

What delegates are you firing?

EventPreSave, EventPostSave, EventFinalSave because I had to use an event to not duplicate core code. I could fire EntryPostCreate and preCreate but that would mean the Event ones would fire as well.

This presents a problem then, and for it to work correctly for you I can't use an event. Surely there must be another way to implement this without duplicating the core.

Edit: Ok, so for the who knows how many-th time, I'll rewrite how the mass upload utility creates entries in the backend, it'll take an hour or two.

Or is this even a big deal, would it be easier to let all 5 events fire?

I feel bad now, if you're going off to rewrite it again

The Reflection Field listens for the EventPostSaveFilter delegate. I can just add the EventPostSave delegate in there, but I don't really know what issues that could cause, if any...

There obviously needs to be something written down in stone for how extensions should listen to and fire delegates, especially ones that run at their own page, rather than the standard pages.

Alright, update the mass upload utility from the master branch @ github. The proper backend events should be firing automatically now. I only briefly tested the bits I changed to make sure it worked, and it did, but let me know if you run across any silly errors.

No big deal, it needs to be written the write way :)

I'll check this when I get home, got my head in some 80,000 records with the XML Importer today...