All parts of a terms declaration (web location, selection, noise removal, distribution across multiple documents…) can change over time. The process of updating these elements to enable continued tracking is called maintenance. Without it, terms can become:
Open Terms Archive needs to keep track of this changes in order to regenerate versions history from snapshots history.
To keep track of services declarations and filters changes, Open Terms Archive offers a versioning system. It is optional and should be added only when needed. It works by creating history files for terms declarations and filters, where each entry should be a previous valid declaration or filter function and should have an expiry date.
Both for terms and filters history, the expiration date is declared in a property validUntil. It should be the authored date and time of the last snapshot commit for which the declaration is still valid.
Terms declarations history files and filters history files can both evolve on their own. Having one does not imply to create the other.
The current (latest) valid declaration has no date and should not appear in the history object: it stays in its own file, just like if there was no history at all.
Declarations history are stored in a history JSON file with the following name declarations/$service_id.history.json.
The terms history contains an object with terms types as properties. Each terms type property is an array of history entries. Each entry has the same format as a normal terms declaration, except there is the mandatory extra property validUntil.
{
…
"<terms type>": [
{
"fetch": "The URL where the document can be found",
"executeClientScripts": "A boolean to execute client-side JavaScript loaded by the document before accessing the content, in case the DOM modifications are needed to access the content; defaults to false (fetch HTML only)",
"filter": "An array of service specific filter function names",
"remove": "A CSS selector, a range selector or an array of selectors that target the insignificant parts of the document that has to be removed. Useful to remove parts that are inside the selected parts",
"select": "A CSS selector, a range selector or an array of selectors that target the meaningful parts of the document, excluding elements such as headers, footers and navigation",
"validUntil": "The inclusive expiration date in ISO format"
}
]
…
}
For example, to add a history entry for the Terms of Service of the service ASKfm, create the file declarations/ASKfm.history.json with the following contents:
{
"Terms of Service": [
{
"fetch": "https://ask.fm/docs/terms_of_use/?lang=en",
"select": "body",
"filter": ["add"],
"validUntil": "2020-10-29T21:30:00.000Z"
}
]
}
Filters history is declared in a filters history declaration JavaScript file with the following name: declarations/$service_id.filters.history.js.
For each filter, a variable named like the filter must be exported. This variable should contain an array of filter history entries. Each entry is an object with the expiration date, as validUntil property, and the valid function for this date, under the filter property. Both properties are mandatory.
export const <filterName> = [
{
validUntil: "The inclusive expiration date in ISO format",
filter: function() { /* body valid until the expiration of the `validUntil` date */ }
}
];
For example, to add a history entry for the removeSharesButton filter of the service ASKfm, create the file declarations/ASKfm.filters.history.js with the following content:
export const removeSharesButton = [
{
validUntil: '2020-08-22T11:30:21.000Z',
filter: async (document) => {
document.querySelectorAll('.shares').forEach((element) => element.remove());
},
},
];
If the service provider stops offering a service, the associated terms will become unavailable. To mark that service termination in Open Terms Archive and ensure tracking tentatives are stopped, while maintaining the possibility to explore the history:
<terms type> entries from the documents key in the declaration:{
"name": "<service name>",
"documents": {}
}
The consensus is to consider that a service provider renaming a service (for example, Twitter to X) is akin to terminating the previous service and opening a new one. Therefore, to apply a service renaming, open a pull request that both terminates the previous service and adds a new service declaration with the new service name. You can reuse the documents part of the original declaration, but should double-check that the selectors and URLs still match, as a service rename is most often accompanied by a new page layout, a new domain name, and sometimes entirely new terms.