3D Digital Engineering Drawings

   Endangered large

3D digital engineering models produced as part of building or engineering design processes. The production of such drawings has progressed from a digital analogue of paper to complex digital environments such as BIM (Building Information Modelling) which combine original drawings, libraries of compound objects, and links to external data sets such as about the performance of materials and maintenance of parts.

Group: Engineering

Trend: No change

Consensus Decision

Added to List: 2017

Last update: 2018

Previous category: Of Concern

Imminence of Action

Action is recommended within three years, detailed assessment within twelve months

Significance of Loss

The loss of tools, data or services within this group would impact on many people and sectors.

Effort to Preserve

It would require a major effort to address losses in this group, possibly requiring the development of new preservation tools or techniques.

Examples

Building Information Management, Computer Aided Design, Product Data Management in engineering and architecture,

‘Critically Endangered’ in the Presence of Aggravating Conditions

Lack of preservation mandate or collecting institution; lack of preservation capability in data owner; irregularities in supply chains; complex or long data supply chains; dependencies on proprietary software or formats; lack of persistent identifiers; poorly managed IPR; temporary joint-venture companies; poor records management; poor regulatory compliance; encryption.

‘Vulnerable’ in the Presence of Good Practice

Well managed data infrastructure; preservation from the point creation; carefully managed IPR; persistent identifiers; well managed records management processes; recognition of preservation requirements at highest levels; strategic investment in digital preservation; host clearly identified; participation in digital preservation community.

2019 Review

This entry was first submitted in 2017 when the Jury lacked the capacity to consider it in detail. It has been assessed this time with additional expertise co-opted. Nonetheless it remains a very broad category including major one-off construction and engineering projects, a long tail of more minor building programmes, as well as large volume but homogenous production processes in engineering. The key consideration is that the lifecycle of the products and the data which describes them vastly exceeds the short lifecycles of the infrastructures on which they are designed. This challenge is compounded by supply chains that may involve many different stages of production, as well as the delivery of large projects through transitory joint ventures companies that have no residual mechanism or capacity to preserve the data thereafter.

Additional Jury Comments

Data in this category enables the safety and security of critical infrastructure, but the responsibility to maintain data is unclear, nor are retention periods clear. Although examples of good practice can be identified, the extent to which there are working solutions at large seems doubtful and it’s surprising that there are not more, and more diverse, success stories to report.

Read More

Born Digital Photographs and Video shared via Social Media or Uploaded to Cloud Services

 

 Endangered large

Digital images or video with no analogue equivalent and where the only copy is online with a social media platform or cloud image hosting service.

Group: Social Media

Trend: No Change

Consensus Decision

Added to List: 2018

Last update: 2019

Previous category: Endangered

Imminence of Action

Action is recommended within three years, detailed assessment within twelve months

Significance of Loss

The loss of tools, data or services within this group would impact on people and sectors around the world.

Effort to Preserve

It would require a major effort to address losses in this group, possibly requiring the development of new preservation tools or techniques.

Examples

Flickr; Vimeo; YouTube; Instagram; Periscope; DropBox; Facebook; Twitter.

‘Critically Endangered’ in the Presence of Aggravating Conditions

lack of preservation capacity in provider; lack of explicit preservation commitment or incentive from provider to preserve; lack of storage replication by provider; dependence on proprietary products or formats; poor management of data protection; inaccessibility to web archiving services; political or commercial interference; lack of offline equivalent; over-abundance; poorly managed intellectual property rights; lossy compression applied in upload scripts.

‘Vulnerable’ in the Presence of Good Practice

Offline backup; lossless compression; good documentation; accessible to web harvesting; clarity of intellectual property rights which enable preservation; credible preservation commitment from service provider; export pathway.

2019 Review

This entry was introduced in 2018 and this time has been placed within a wider social media group. It shares many of the challenges with other entries in the group, with a dependency on a global service provider whose business model can only be presumed and tied to users via asymmetrical contracts that favour the supplier. Moreover, because these services have a low barrier to entry they are used by agencies or individuals least able to respond to closure or loss. Images and videos present distinct preservation challenges which need to be addressed.

Additional Jury Comments

The platforms themselves get in the way. If there were enough money and an institution willing to take on large datasets or archiving via existing web archiving tools, with the permission of rights holders and the platforms it would probably be possible to harvest this content. This gets trickier with the closed networks and content in user accounts that are not accessible to the public, so web archives wouldn't necessarily be able to step in. Instead, individual users armed with personal digital archiving tools and knowledge would need to preserve their own data.

Read More

Completed investigations based on open source intelligence sources

 

 Endangered large

Open source social media and web content that has been used to support the conclusions of crowd-sourced investigation and fact-checking in political or military conflict.

Group: Digital Legal Records

Trend: New Entry

Unanimous Decision

Added to List: 2019

Last update: 2019

Previous category: New Entry

Imminence of Action

Action is recommended within three years, detailed assessment within twelve months

Significance of Loss

The loss of tools, data or services within this group would impact on people and sectors around the world.

Effort to Preserve

It would require a small effort to preserve materials in this group, with the deployment of proven tools or techniques.

Examples

Social media sources relating to recent conflicts, such as in Ukraine.

Critically Endangered in the presence of Aggravating Conditions

Encryption; loss of authenticity; lack of preservation agency; limited or no digital preservation capability.

‘Vulnerable’ in the Presence of Good Practice

Offline backup captured by journalist or investigating authority; materials presented and documented in court; court able to deliver preservation; authenticity protected

2019 Review

This is a new entry received through open submission in 2019 and subsequently split into three elements by the Jury, relating to current, recent and historic sources.  This entry relates to materials used in evidence in completed investigations, as well as those presented to courts or other investigatory agencies. 

Social media companies have a policy to take down or suppress content that they consider to be propaganda for terrorist groups.  This has had the unintended consequence of deleting or supressing content that was being used in open source investigation or fact checking for journalistic or judicial purposes, and which may therefore be an impediment to refutation or prosecution.  However, a new generation of cloud based services now allow investigators to copy and stabilise content to private accounts in the process of investigating it: so the ethical requirements of social media companies and the integrity of investigation are both served.

The Jury notes that the such content remains at risk, but the presentation of data to a court or prosecuting authority, or the publication through news media implies the introduction of a long-term preservation function. 

Additional Jury Comments

-

Read More

Consumer cloud-based utilities

   Endangered large

Free online utilities offered at no cost to end users, but with a business model based on gathering and reselling consumer insights.  This entry includes data produced and stored and accessed within the utilities.

Group: Social media

Trend: New Entry

Consensus Decision

Added to List: 2019

Last update: 2019

Previous category: New Entry

Imminence of Action

Action is recommended within three years, detailed assessment within twelve months

Significance of Loss

The loss of tools, data or services within this group would impact on people and sectors around the world.

Effort to Preserve

It would require a major effort to address losses in this group, possibly requiring the development of new preservation tools or techniques.

Examples

Google Docs, Google Sheets, Gmail, Hotmail, Yahoo Mail, Prezi

‘Critically Endangered’ in the Presence of Aggravating Conditions

Unstable business model from service provider; service provider bought over or pivots to new market opportunities; lack of investment in infrastructure; lack of offline copy.

Vulnerable in the Presence of Good Practice

Clear export and migration pathways; preservation responsibility shouldered by the service provider;

2019 Review

The judges have introduced this entry as a subset of a previous entry, emphasizing the different threats faced by services that are ‘paid-for’ versus ‘free-at-the-point-of-use’ tools. Both depend on the business model of the vendor and the terms and conditions which they impose. This group of digital materials the business model and sustainability can only be presumed and contracts tend to be asymmetrical in favour of the supplier. Moreover, because these services have a low barrier to entry they may be favoured by agencies or individuals least able to respond to closure or loss.

Additional Jury Comments

If we're talking about the entire platforms and risk of the entirety of data on these, then the concern is that the corporation providing the service suddenly decides it's no longer of value to them. In these circumstances material could be removed quickly. That has happened already with some services and will certainly be seen again. Preservation is not a commitment that most providers make. Cultural Heritage organizations need to determine what out of these services is important to tackle and may use traditional web archiving tools or email preservation tools to get what they can. Possibly new tools around preserving data in these services for particular accounts - a sort of personal digital archive option – could be helpful and encouraged.

Read More

Consumer social media free at the point of use

   Endangered large

Social media services offered free at the point of use with a subscription model based on reselling user behavior and/or advertising.

Group: Social Media

Trend: No change

Consensus Decision

Added to List: 2017

Last update: 2018

Previous category: Endangered

Imminence of Action

Action is recommended within three years, detailed assessment within twelve months

Significance of Loss

The loss of tools, data or services within this group would impact on people and sectors around the world

Effort to Preserve

It would require a major effort to address losses in this group, possibly requiring the development of new preservation tools or techniques.

Examples

Flickr, Vimeo, YouTube, Instagram, Periscope, DropBox, Facebook, Twitter, Pinterest, Yahoo Groups

‘Critically Endangered’ in the Presence of Aggravating Conditions

Lack of preservation capacity in provider; Lack of preservation commitment or incentive from provider; Lack of storage replication; proprietary products or formats; poor data protection; inaccessibility to web archiving; political or commercial interference; Lack of offline equivalent; super-abundance; poorly managed IPR; Lossy compression in upload scripts.

Vulnerable in the Presence of Good Practice

Offline backup and documentation of media assets; Migration plan; Early warning from vendors; Roadmap from vendors; Accessible to web harvest; Suitable export functionality; Licencing enables preservation; Preservation commitment from vendor; Preservation capability in vendor;

Resilient to hacking; Selection criteria;

2019 Review

The judges have introduced this entry as a subset of a previous entry, emphasizing the different threats faced by online services that are ‘paid-for’ versus ‘free-at-the-point-of-use’. Both depend on the business model of the vendor and the terms which they impose. For this group of digital materials the business model and sustainability can only be guessed and contracts tend to be asymmetrical in favour of the supplier. Moreover, because these services have a low barrier to entry they may be favoured by agencies or individuals least able to respond to closure or loss.

Additional Jury Comments

Social media capture via web harvesting has become increasingly difficult. The platforms continue to put up barriers to automated capture that prevent preservation of even so-called public content. For example, campaign websites or other election-related content that is only published in Facebook or on Twitter, because these services are "Free". This content is of particular concern as it appears on no other website. Web archivists are constantly shifting strategies and approaches and trying out new (but limited) tools to best capture this content. If we cannot successfully preserve these platforms, we are missing out on documenting organizations, campaigns and elections around the globe. Much of this data exists as data sets based on aggregated use rather than individual files.

Read More

Contractual Documents and Related Records

   Endangered large

Documents, correspondence and other records created in the course of contractual dealings between individuals and agencies, especially where the subjects are of long duration and may be subject to legal scrutiny at undefined points in the distant future.

Group: Digital Legal Records

Trend: No change

Consensus Decision

Added to List: 2017

Last update: 2018

Previous category: Endangered

Imminence of Action

Action is recommended within three years, detailed assessment within twelve months

Significance of Loss

The loss of tools, data or services within this group would impact on many people and sectors.

Effort to Preserve

It would require a small effort to preserve materials in this group, with the deployment of proven tools or techniques.

Examples

Contracts, receipts, correspondence, license agreements, building consent, warranties, and any other document or record that represents a legally binding transaction or permission. Such records may be useful in the avoidance or resolution of disputes whether in court or prior to proceedings. Includes Online Terms and Conditions for e-commerce or end-user agreements for services.

‘Critically Endangered’ in the Presence of Aggravating Conditions

Loss of context; loss of authenticity or integrity; external dependencies; poor storage; lack of understanding; churn of staff; poorly framed or over-zealous disposal; ill-informed records management; misplaced fears with respect to data protection, encryption.

Vulnerable in the Presence of Good Practice

Well managed data infrastructure; preservation enabled at the point of creation; carefully managed authenticity; use of persistent identifiers; finding aids; well managed records management processes; application of records management standards

2019 Review

This entry is a subset of an entry made in 2017 for ‘Digital Legal Records and Evidence’ which the Jury has decided to split into four more discrete entries this year. It has significant overlap with a separate entry about Pension Mortgage and Insurance Records, except that those are created explicitly with the expectation that they are tied to long-lifecycle products and are held by long lived agencies who provide those products. This entry notes that many digital records have value over the long term which may not be immediately obvious at the point of creation.

Additional Jury Comments

Legal agreements should be dealt with through records management standards and processes with agencies who have experience of handling this.

Read More

Corporate Records of Long Duration on Network Drives, Intranets and EDRMS

   Endangered large

Records on internal corporate network drives, intranets or document management services where access is limited to a distinct group of users, and in which the lifecycle of the record or the business processes they support is greater than the technology on which they are created or retained.

Group: Sensitive Data

Trend: No Change

Consensus Decision

Added to List: 2017

Last update: 2018

Previous category: Endangered

Imminence of Action

Action is recommended within three years, detailed assessment within twelve months

Significance of Loss

The loss of tools, data or services within this group would impact on many people and sectors.

Effort to Preserve

It would require a major effort to preserve materials in this group, with the development of new preservation tools or techniques.

Examples

Born digital records of small and medium sized enterprises; fasting-changing internal manuals, advice or policies shared on intranets or EDRMS; records of long-lived products and services; Historic guidelines and manuals which evidence 'best practice'; Documentation supporting long-lived contractual relations; Online terms and conditions; Corporate Slack channels; Google Drives; EDRMS; Email.

‘Critically Endangered’ in the Presence of Aggravating Conditions

Lack of systematic preservation function; lack of preservation path or plan for data; dependence on proprietary products or formats; poor management of data protection; political or commercial interference; lack of offline equivalent; over-abundance through poor disposal or version control; lack of capacity; lack of commitment; loss or lack of documentation; sector specific software or data types; encryption

‘Vulnerable’ in the Presence of Good Practice

Preservation infrastructure and pathways; replication; appraisal and selection including de-duplication;

2019 Review

This entry draws attention to the pressing need for digital preservation in business, especially in small to medium enterprises.

Additional Jury Comments

Corporate records should form part of organisational records management schemes and so responsibilities should be clear, however this may be much more challenging for smaller organisations without dedicated roles, or with complex data types.

Read More

Current Portable Magnetic Media

   Endangered large

Materials saved to magnetic tape, portable hard disks or other magnetic media in the last five years where the reader devices are still supported and can be integrated easily into hardware infrastructure.

Group: Portable Media

Trend: new entry

Consensus Decision

Added to List: 2019

Last update: 2019

Previous category: New Entry

Imminence of Action

Action is recommended within three years, detailed assessment within twelve months

Significance of Loss

The loss of tools, data or services within this group would impact on many people and sectors.

Effort to Preserve

It would require a small effort to address losses in this group, requiring the application of proven preservation tools or techniques.

Examples

LTO8 tapes; portable hard disks

‘Critically Endangered’ in the Presence of Aggravating Conditions

Poor storage conditions; encryption; digital rights management; lack of replication; lack of documentation; lack of periodic testing; lack of refreshment pathway; lack of access to readers; out of manufacturers’ warranty or no warranty; storage within paper files.

Vulnerable in the Presence of Good Practice

Regular review and testing; replication; refreshment plan; comprehensive documentation; high quality storage; regular maintenance of readers; multiple readers available.

2019 Review

The judges introduced this entry to ensure that the range of media storage is properly assessed and presented. Magnetic media is typically more fragile than optical media because it is susceptible to ‘bitrot’ and magnetic damage in ways that optical media are not.

Additional Jury Comments

This entry is highly dependent on who is looking after the portable media but made more difficult over time. The lack of granularity in the definition means that only general advice can be offered such as to refresh media. In time it may yet be more useful to split all storage media (maybe 100 items long) with an indication of how long these can be expected to last. In many cases obsolete media can be recovered by specialist companies but the cost of employing them can become an aggravating condition.

It’s important to emphasize that the short lifetime of many storage devices is not a problem to be solved with new long-lasting storage technologies (and indeed, many inventions have come and gone). Cheap, commodity storage has been purposely designed to deliver value at a low price for a short time, and therefore management and preservation processes for monitoring and refreshment need to take these characteristics into account.

Read More

Current Portable Optical Media

   Endangered large

Materials saved to DVDs, CDs or other optical media in the last five years where the reader devices are still supported and can be integrated easily into hardware infrastructure

Group: Portable Media

Trend: New Entry

Consensus Decision

Added to List: 2019

Last update: 2019

Previous category: New Entry

Imminence of Action

Action is recommended within three years, detailed assessment within twelve months

Significance of Loss

The loss of tools, data or services within this group would impact on many people and sectors.

Effort to Preserve

It would require a small effort to address losses in this group, requiring the application of proven preservation tools or techniques.

Examples

CDs, DVDs produced in the last five years

‘Critically Endangered’ in the Presence of Aggravating Conditions

Poor storage conditions; encryption; digital rights management; lack of replication; lack of documentation; lack of periodic testing; lack of refreshment pathway; lack of access to readers; out of manufacturers’ warranty or no warranty; storage in paper files.

Vulnerable in the Presence of Good Practice

Regular review and testing; replication; refreshment plan; comprehensive documentation; high quality storage; regular maintenance of readers; multiple readers available;

2019 Review

The judges have introduced this entry to ensure that the range of media storage is properly assessed and presented.

Additional Jury Comments

This entry is highly dependent on who is looking after the portable media but made more difficult over time. The lack of granularity in the definition means that only general advice can be offered such as to refresh media. In time it may yet be more useful to split all storage media (maybe 100 items long) with an indication of how long these can be expected to last. In many cases obsolete media can be recovered by specialist companies but the cost of employing them can become an aggravating condition.

Read More

Current Portable Solid-State Media

   Endangered large

Materials saved to flash drives or other solid-state media in the last five years where the reader devices are still supported and can be integrated easily into hardware infrastructure

Group: Portable Media

Trend: New Entry

Consensus Decision

Added to List: 2019

Last update: 2019

Previous category: New Entry

Imminence of Action

Action is recommended within three years, detailed assessment within twelve months

Significance of Loss

The loss of tools, data or services within this group would impact on many people and sectors.

Effort to Preserve

It would require a small effort to address losses in this group, requiring the application of proven preservation tools or techniques.

Examples

USB memory sticks; flash cards in cameras; solid state portable hard disks

‘Critically Endangered’ in the Presence of Aggravating Conditions

Poor storage conditions; encryption; digital rights management; lack of replication; lack of documentation; lack of periodic testing; lack of refreshment pathway; lack of access to readers; out of manufacturers’ warranty or no warranty; storage in paper files.

Vulnerable in the Presence of Good Practice

Regular review and testing; replication; refreshment plan; comprehensive documentation; high quality storage; regular maintenance of readers; multiple readers available;

2019 Review

The judges have introduced this entry to ensure that the range of media storage is properly assessed and presented.

Additional Jury Comments

This entry is highly dependent on who is looking after the portable media but made more difficult over time. The lack of granularity in the definition means that only general advice can be offered such as to refresh media. In time it may yet be more useful to split all storage media (maybe 100 items long) with an indication of how long these can be expected to last. In many cases obsolete media can be recovered by specialist companies but the cost of employing them can become an aggravating condition.

Read More

Scroll to top