Recommended practices – Part-1: Storage of unstructured data

This is a part one of a multi-part series on recommended practices for organizations and their end-users. Additional parts will be in upcoming newsletters.

Organizations create and consume data constantly, but not all have formal policies or practices that define the value of this data and restrict its amount and location.

Quality is difficult to define and even tougher to enforce; some departments and users save items solely for convenience, even though its value is minimal, while others consider everything they have ever said or done, even 20 years ago, to be worthy of permanent storage. Basically, there is no point to storing unstructured data (MS Office documents, PDFs, etc.) unless it has value to the organization; however, if you must store it, choose a method that allows some type of classification (like SharePoint with its searchable repository of metadata).1

Rather than try to enforce quality standards, many organizations impose limitations on the amount of data stored (since this can be controlled and monitored)2: Even though disk space is relatively inexpensive, backup, data-management, and data-security costs increase as data grows. Quotas also impose discipline; setting a quota allows the organization to get a picture of storage needs by individuals and by departments or functional groups. Quotas can also be adjusted as needed.

There are tools that manage unstructured data via audit/access controls and monitor via usage patterns; these are targeted (and priced) for enterprise-class organizations, but are moving downstream within the reach of more organizations. There are less-expensive tools (and policies included within Active Directory) that limit storage-space usage; limits are usually set by user or by department.

Finally, organizations traditionally assume, and try to enforce, that end-users save and store company data only at designated locations of on-premise equipment (drives mapped to servers, storage arrays, Network Attached Storage, etc.) or at authorized, Cloud-based storage locations; the idea is to save and secure company data where it will receive proper backup, security, and vetting. Saving company data onto personal computers, tablets, and mobile phones, where it might not receive regular backups and is more vulnerable to loss or theft, is discouraged.

The best place to start is to create a clear, unambiguous policy on the storage of company data with these guidelines:

Define what data should be kept and for how long
Define storage-amount limitations and enforcement
Define acceptable storage locations
Define responsibilities for retention
Once defined, processes can be created and tools can be acquired to manage and monitor this policy.

Our recommendations for storage locations:

Remove all data from end-user devices (laptops, mobile devices, etc.).
Map a Home folder for each end-user and restrict its rights to that user.
Move the end-user My documents folder to their respective Home folder.
Deploy a document-collaboration utility (like SharePoint or Google Docs) or create a Shared folder with appropriate subfolders to manage your shared, unstructured data.
Restrict shared access by department or functional group.
Our recommendations for storage management:

Define policies within Active Directory to limit storage space (as needed).
Archive older, infrequently-used data to less-expensive storage.
Monitor usage on a regular basis.
1. Visit “My ongoing rant about unstructured end user data storage”.

2. See Alan Radding’s excellent and relevant article “Keep end-user storage under control” at TechTarget and originally from Storage magazine in November 2006.