Attention users of mm_media

classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Attention users of mm_media

Dan Wilga-2
I have been spending a lot of time recently trying to unravel a mystery surrounding nodes with files, both as mm_media nodes and articles with attachments. First, the symptom:

SELECT * FROM file_usage f INNER JOIN file_managed m ON m.fid = f.fid LEFT JOIN node n ON n.nid = f.id WHERE n.nid IS NULL AND f.type='node';

This query shows that we (and maybe you) have binary files that are associated with nodes that no longer exist.

I believe I have isolated one case of this, and it has to do with the switch we made some time ago in mm_media. When we did that as part of the D6->7 upgrade, we began using a mediafield field to hold the data rather than doing it internally
in mm_media's own tables. But the conversion hook created entries in the file_usage table using the "mm_media" module, which it should not have. Instead, it should have used "mediafield" as the module name.

As a result of this bug, the following steps will induce orphan rows in file_usage and file_managed:

1. SELECT id AS nid FROM file_usage f INNER JOIN node n ON n.nid = f.id WHERE module='mm_media';
2. Take one of these nodes and put it on a page by itself.
3. Use MM's copy feature to copy (with contents) the page to another location. This duplicates the node, but creates another reference to the same binary in file_usage.
4. Delete the original node (with node_delete(), for instance.)

The result is that both entries remain in file_usage, even though the original node has been deleted.

So, here's what I think I need to do:

1. Write an update hook to change all file_usage=='mm_media' to 'mediafield'.
2. Remove mm_media_node_delete(), since it's not necessary. Its function happens automatically in mediafield_field_delete().

3. Here's the dangerous part: Call file_delete() for each file in:

 
SELECT fid FROM file_usage f LEFT JOIN node n ON n.nid = f.id WHERE n.nid IS NULL AND f.type='node';

I hesitate to put this last step into an update hook, though, since it's so destructive. What do you think? It's removing binaries that are no longer associated with nodes, so they aren't visible anywhere, but it still seems wrong to do without warning.

======

The other issue is that I am still seeing a number of orphans that are not explained by the above cause:

SELECT * FROM file_usage f INNER JOIN file_managed m ON m.fid = f.fid LEFT JOIN node n ON n.nid = f.id WHERE n.nid IS NULL AND f.type='node' AND module<>'mm_media';

We have 311 items, all with the module=='file' indicating they are attachments onto articles, that are orphans. I suppose this sort of thing could happen if node deletion was interrupted prematurely, but I'm curious how many of you have things like this.

---

You are currently subscribed to monster_menus as: [hidden email].

To unsubscribe click here: http://lists.middlebury.edu/u?id=685503.6b071f880fe6a965a128164e6d09ea81&n=T&l=monster_menus&o=718351

(It may be necessary to cut and paste the above URL if the line is broken)

or send a blank email to [hidden email]

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

RE: Attention users of mm_media

Hilary Caws-Elwitt

Hi Dan and all,

 

We have 228 orphan files, all with module=’file’ (we were never on D6, so presumably immune from your first category). Most of the NIDS are quite old, so I wondered if this might have been related somehow to the recycle bin issues that were fixed in 7.x-1.18, but I have at least one that was created after we upgraded. I’ll do some more digging if I get a chance.

 

--

Hilary Caws-Elwitt

IT Analyst - Five Colleges, Inc. - http://www.fivecolleges.edu

97 Spring St, Amherst MA 01002

[hidden email] - 413-542-4022

 

From: Dan Wilga [mailto:[hidden email]]
Sent: Thursday, July 16, 2015 11:20 AM
To: Monster Menus Development
Subject: Attention users of mm_media

 

I have been spending a lot of time recently trying to unravel a mystery surrounding nodes with files, both as mm_media nodes and articles with attachments. First, the symptom:

SELECT * FROM file_usage f INNER JOIN file_managed m ON m.fid = f.fid LEFT JOIN node n ON n.nid = f.id WHERE n.nid IS NULL AND f.type='node';

This query shows that we (and maybe you) have binary files that are associated with nodes that no longer exist.

I believe I have isolated one case of this, and it has to do with the switch we made some time ago in mm_media. When we did that as part of the D6->7 upgrade, we began using a mediafield field to hold the data rather than doing it internally in mm_media's own tables. But the conversion hook created entries in the file_usage table using the "mm_media" module, which it should not have. Instead, it should have used "mediafield" as the module name.

As a result of this bug, the following steps will induce orphan rows in file_usage and file_managed:

1. SELECT id AS nid FROM file_usage f INNER JOIN node n ON n.nid = f.id WHERE module='mm_media';
2. Take one of these nodes and put it on a page by itself.
3. Use MM's copy feature to copy (with contents) the page to another location. This duplicates the node, but creates another reference to the same binary in file_usage.
4. Delete the original node (with node_delete(), for instance.)

The result is that both entries remain in file_usage, even though the original node has been deleted.

So, here's what I think I need to do:

1. Write an update hook to change all file_usage=='mm_media' to 'mediafield'.
2. Remove mm_media_node_delete(), since it's not necessary. Its function happens automatically in mediafield_field_delete().
3. Here's the dangerous part: Call file_delete() for each file in:

  SELECT fid FROM file_usage f LEFT JOIN node n ON n.nid = f.id WHERE n.nid IS NULL AND f.type='node';

I hesitate to put this last step into an update hook, though, since it's so destructive. What do you think? It's removing binaries that are no longer associated with nodes, so they aren't visible anywhere, but it still seems wrong to do without warning.

======

The other issue is that I am still seeing a number of orphans that are not explained by the above cause:

SELECT * FROM file_usage f INNER JOIN file_managed m ON m.fid = f.fid LEFT JOIN node n ON n.nid = f.id WHERE n.nid IS NULL AND f.type='node' AND module<>'mm_media';


We have 311 items, all with the module=='file' indicating they are attachments onto articles, that are orphans. I suppose this sort of thing could happen if node deletion was interrupted prematurely, but I'm curious how many of you have things like this.

---

You are currently subscribed to monster_menus as: [hidden email].

To unsubscribe click here: http://lists.middlebury.edu/u?id=1034715.d8dc340b0014c740c37e95754e54e1f3&n=T&l=monster_menus&o=718351

(It may be necessary to cut and paste the above URL if the line is broken)

or send a blank email to [hidden email]

---

You are currently subscribed to monster_menus as: [hidden email].

To unsubscribe click here: http://lists.middlebury.edu/u?id=685503.6b071f880fe6a965a128164e6d09ea81&n=T&l=monster_menus&o=718413

(It may be necessary to cut and paste the above URL if the line is broken)

or send a blank email to [hidden email]

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Attention users of mm_media

Dan Wilga-2
In reply to this post by Dan Wilga-2
I have a revised version of the suggested process. For this query:

  SELECT fid FROM file_usage f LEFT JOIN node n ON n.nid = f.id WHERE f.type='node' GROUP BY f.fid HAVING n.nid IS NULL;

call file_delete(fid, TRUE);

Then, this query:

  DELETE f FROM file_usage f LEFT JOIN node n ON n.nid = f.id WHERE n.nid IS NULL AND f.type='node';

The first query finds orphan rows for which the file is not being used by another node and deletes their file_managed and file_usage entries. The second query finds any remaining orphans and removes just their file_usage entries, since they are still being used by another node.

On 7/16/15 11:20 AM, Dan Wilga wrote:
I have been spending a lot of time recently trying to unravel a mystery surrounding nodes with files, both as mm_media nodes and articles with attachments. First, the symptom:

SELECT * FROM file_usage f INNER JOIN file_managed m ON m.fid = f.fid LEFT JOIN node n ON n.nid = f.id WHERE n.nid IS NULL AND f.type='node';

This query shows that we (and maybe you) have binary files that are associated with nodes that no longer exist.

I believe I have isolated one case of this, and it has to do with the switch we made some time ago in mm_media. When we did that as part of the D6->7 upgrade, we began using a mediafield field to hold the data rather than doing it internally
in mm_media's own tables. But the conversion hook created entries in the file_usage table using the "mm_media" module, which it should not have. Instead, it should have used "mediafield" as the module name.

As a result of this bug, the following steps will induce orphan rows in file_usage and file_managed:

1. SELECT id AS nid FROM file_usage f INNER JOIN node n ON n.nid = f.id WHERE module='mm_media';
2. Take one of these nodes and put it on a page by itself.
3. Use MM's copy feature to copy (with contents) the page to another location. This duplicates the node, but creates another reference to the same binary in file_usage.
4. Delete the original node (with node_delete(), for instance.)

The result is that both entries remain in file_usage, even though the original node has been deleted.

So, here's what I think I need to do:

1. Write an update hook to change all file_usage=='mm_media' to 'mediafield'.
2. Remove mm_media_node_delete(), since it's not necessary. Its function happens automatically in mediafield_field_delete().

3. Here's the dangerous part: Call file_delete() for each file in:

 
SELECT fid FROM file_usage f LEFT JOIN node n ON n.nid = f.id WHERE n.nid IS NULL AND f.type='node';

I hesitate to put this last step into an update hook, though, since it's so destructive. What do you think? It's removing binaries that are no longer associated with nodes, so they aren't visible anywhere, but it still seems wrong to do without warning.

======

The other issue is that I am still seeing a number of orphans that are not explained by the above cause:

SELECT * FROM file_usage f INNER JOIN file_managed m ON m.fid = f.fid LEFT JOIN node n ON n.nid = f.id WHERE n.nid IS NULL AND f.type='node' AND module<>'mm_media';

We have 311 items, all with the module=='file' indicating they are attachments onto articles, that are orphans. I suppose this sort of thing could happen if node deletion was interrupted prematurely, but I'm curious how many of you have things like this.

---

You are currently subscribed to monster_menus as: [hidden email].

To unsubscribe click here: http://lists.middlebury.edu/u?id=685500.19fa7de7038497527f6a88cf1629251d&n=T&l=monster_menus&o=718351

(It may be necessary to cut and paste the above URL if the line is broken)

or send a blank email to [hidden email]


---

You are currently subscribed to monster_menus as: [hidden email].

To unsubscribe click here: http://lists.middlebury.edu/u?id=685503.6b071f880fe6a965a128164e6d09ea81&n=T&l=monster_menus&o=718509

(It may be necessary to cut and paste the above URL if the line is broken)

or send a blank email to [hidden email]

Loading...