Maintenance - Revision history

Sam: /* Delete Media */

2023-04-30T17:46:27Z

Delete Media

Sam: /* Why */

2023-03-02T10:15:45Z

Why

Sam at 21:27, 25 July 2022

2022-07-25T21:27:36Z

Sam: /* Delete Media */

2022-07-25T20:30:07Z

Delete Media

Sam: /* Why */

2022-07-25T20:24:18Z

Why

Sam: /* Why */

2022-07-25T20:16:39Z

Why

Sam: /* Delete Media */

2022-07-25T19:53:19Z

Delete Media

Sam at 19:40, 25 July 2022

2022-07-25T19:40:27Z

Sam at 17:45, 25 July 2022

2022-07-25T17:45:54Z

Sam at 17:37, 25 July 2022

2022-07-25T17:37:39Z

@@ Line 16: / Line 16: @@
 This process may take a while so starting it on a Friday afternoon and letting it run over the weekend might be a good idea.
-It is far more efficient to delete multiple media before executing a [[Truxton Status#Tray Menu|"Clean Data"]] because it increases the chances that depot files can be simply deleted rather than have their contents consolidated multiple times.
+It is far more efficient to delete multiple media before executing a [[Truxton Status#Tray Menu|"Clean Data"]] because it increases the chances that depot files can be deleted rather than have their contents consolidated multiple times.
 ==Why==

@@ Line 29: / Line 29: @@
 Deleting both the metadata for a piece of media and its contents results in a workflow of:
 # Delete the records from the database for the given media identifier
-# Remove records from the <code>[Content]</code> table that no longer have a hash that exists in the <code><nowiki>[</nowiki>[[File Table|File]]<nowiki>]</nowiki></code> table
+# Remove records from the <code><nowiki>[</nowiki>[[Content Table|Content]]<nowiki>]</nowiki></code> table that no longer have a hash that exists in the <code><nowiki>[</nowiki>[[File Table|File]]<nowiki>]</nowiki></code> table
-# Remove records from the <code>[Depot]</code> table that are no longer referenced in the <code>[Content]</code> and rename the depot file to include <code>.ToBeDeleted</code>
+# Remove records from the <code><nowiki>[</nowiki>[[Depot Table|Depot]]<nowiki>]</nowiki></code> table that are no longer referenced in the <code><nowiki>[</nowiki>[[Content Table|Content]]<nowiki>]</nowiki></code> and rename the depot file to include <code>.ToBeDeleted</code>
 # Consolidate depots to remove the no longer referenced contents and rename donor depots to <code>.ToBeDeleted</code>
 # Delete the <code>.ToBeDeleted</code> depot files

← Older revision		Revision as of 21:27, 25 July 2022
Line 42:		Line 42:
	You can stop the consolidation process by shutting down the [[Maintenance ETL]].		You can stop the consolidation process by shutting down the [[Maintenance ETL]].
	When it restarts, it will pick up the next message in its queue.		When it restarts, it will pick up the next message in its queue.
		+
		+	=Slowing Down=
		+	As Truxton grows, the database has more work to do.
		+	Often times, the query planner will have outdated statistics to deal with.

@@ Line 15: / Line 15: @@
 By selecting [[Truxton Status#Tray Menu|"Clean Data"]] from the Truxton Tray application, you will enqueue tasks to [[Maintenance ETL#Clean Database|clean]] the database of orphaned records, [[Maintenance ETL#Consolidate Depots|consolidate]] the depot files, [[Maintenance ETL#Consolidate Depots|Delete Depots]] unused depot files, and [[Maintenance ETL#Optimize|optimize]] the database.
 This process may take a while so starting it on a Friday afternoon and letting it run over the weekend might be a good idea.
 ==Why==

← Older revision		Revision as of 20:24, 25 July 2022
Line 31:		Line 31:
	# Consolidate depots to remove the no longer referenced contents and rename donor depots to <code>.ToBeDeleted</code>		# Consolidate depots to remove the no longer referenced contents and rename donor depots to <code>.ToBeDeleted</code>
	# Delete the <code>.ToBeDeleted</code> depot files		# Delete the <code>.ToBeDeleted</code> depot files
		+
		+	The good news is users do not have to wait for this process to finish.
		+	Access to file contents is never impeded.
		+	The bad news is this process cannot be parallelized.
		+	It requires a single process to handle the movement of the contents between depot files.
		+	When the [[Maintenance ETL]] is busy consolidating depots, it cannot process any other messages.
		+	Its message queue will fill with work to be done.
		+	You can stop the consolidation process by shutting down the [[Maintenance ETL]].
		+	When it restarts, it will pick up the next message in its queue.

← Older revision		Revision as of 20:16, 25 July 2022
Line 24:		Line 24:
	The simplicity of one depot per ETL comes at the cost of contents from different media existing in the same depot file.		The simplicity of one depot per ETL comes at the cost of contents from different media existing in the same depot file.
	Complexity is introduced only when you want to remove contents from Truxton.		Complexity is introduced only when you want to remove contents from Truxton.
		+
		+	Deleting both the metadata for a piece of media and its contents results in a workflow of:
		+	# Delete the records from the database for the given media identifier
		+	# Remove records from the <code>[Content]</code> table that no longer have a hash that exists in the <code><nowiki>[</nowiki>[[File Table\|File]]<nowiki>]</nowiki></code> table
		+	# Remove records from the <code>[Depot]</code> table that are no longer referenced in the <code>[Content]</code> and rename the depot file to include <code>.ToBeDeleted</code>
		+	# Consolidate depots to remove the no longer referenced contents and rename donor depots to <code>.ToBeDeleted</code>
		+	# Delete the <code>.ToBeDeleted</code> depot files

← Older revision		Revision as of 19:53, 25 July 2022
Line 14:		Line 14:
	To do that, you must perform a [[Maintenance ETL#Consolidate Depots\|depot consolidation]].		To do that, you must perform a [[Maintenance ETL#Consolidate Depots\|depot consolidation]].
	By selecting [[Truxton Status#Tray Menu\|"Clean Data"]] from the Truxton Tray application, you will enqueue tasks to [[Maintenance ETL#Clean Database\|clean]] the database of orphaned records, [[Maintenance ETL#Consolidate Depots\|consolidate]] the depot files, [[Maintenance ETL#Consolidate Depots\|Delete Depots]] unused depot files, and [[Maintenance ETL#Optimize\|optimize]] the database.		By selecting [[Truxton Status#Tray Menu\|"Clean Data"]] from the Truxton Tray application, you will enqueue tasks to [[Maintenance ETL#Clean Database\|clean]] the database of orphaned records, [[Maintenance ETL#Consolidate Depots\|consolidate]] the depot files, [[Maintenance ETL#Consolidate Depots\|Delete Depots]] unused depot files, and [[Maintenance ETL#Optimize\|optimize]] the database.
		+	This process may take a while so starting it on a Friday afternoon and letting it run over the weekend might be a good idea.
		+
		+	==Why==
		+	The complete deletion of media is a multi-stage process because of Truxton's scalable design.
		+	Writing to a depot file is isolated to a single process.
		+	There can be hundreds of ETL processes running on many machines throughout the network.
		+	By letting each ETL process have its own depot file to write to, no coordination with other processes is needed and performance improved.
		+	But, the [https://en.wikipedia.org/wiki/There_ain%27t_no_such_thing_as_a_free_lunch TANSTAAFL] principal states that you have to pay for it somewhere.
		+	The simplicity of one depot per ETL comes at the cost of contents from different media existing in the same depot file.
		+	Complexity is introduced only when you want to remove contents from Truxton.

@@ Line 12: / Line 12: @@
 These will delete the meta data pertaining to a piece of media from the database.
 It will not delete the file contents associated with the media.
-To do that, you must perform a depot consolidation.
+To do that, you must perform a [[Maintenance ETL#Consolidate Depots|depot consolidation]].
-By selecting [[Truxton Status#Tray Menu|"Clean Data"]] from the Truxton Tray application, you will enqueue tasks to clean the database of orphaned records, consolidate the depot files, delete unused depot files, and optimize the database.
+By selecting [[Truxton Status#Tray Menu|"Clean Data"]] from the Truxton Tray application, you will enqueue tasks to [[Maintenance ETL#Clean Database|clean]] the database of orphaned records, [[Maintenance ETL#Consolidate Depots|consolidate]] the depot files, [[Maintenance ETL#Consolidate Depots|Delete Depots]] unused depot files, and [[Maintenance ETL#Optimize|optimize]] the database.

← Older revision		Revision as of 17:45, 25 July 2022
Line 13:		Line 13:
	It will not delete the file contents associated with the media.		It will not delete the file contents associated with the media.
	To do that, you must perform a depot consolidation.		To do that, you must perform a depot consolidation.
		+	By selecting [[Truxton Status#Tray Menu\|"Clean Data"]] from the Truxton Tray application, you will enqueue tasks to clean the database of orphaned records, consolidate the depot files, delete unused depot files, and optimize the database.
		+
		+	==Killing Orphans==
		+
		+	==Depot Consolidation==
		+
		+	==Deleting Depot Files==
		+
		+	==Optimize the Database==

← Older revision		Revision as of 17:37, 25 July 2022
Line 4:		Line 4:

	=Delete Media=		=Delete Media=
		+	When you want to delete a piece of media from Truxton, you have a few choices:
		+
		+	# Use the Media Management page on the Analyst Desktop
		+	# Use [[TruxtonCLI.exe#Delete a Piece of Media\|TruxtonCLI]]
		+	# Write your own [[Truxton_delete_media\|program]] or [[TruxtonObject#deletemedia.28media_id.29\|script]]
		+
		+	These will delete the meta data pertaining to a piece of media from the database.
		+	It will not delete the file contents associated with the media.
		+	To do that, you must perform a depot consolidation.