PAT.File Storage

Informational

Page maturity
This page has maturity level 4 (mature)

	PAT	File Storage	Version:	0.4
	Document type:	Pattern Type	Owner:	J.A.H. Schoonderbeek

Informational

Create commentary

Description

This Pattern Type belongs to "Infrastructure Sector Core". This facility offers data preservation and data retrieval functionality with the following characteristics:

operates on data at the "loosely structured data" level, e.g. files or documents,
located at one logical location,
may offer the ability to perform storage related "intelligent" tasks, such as replication, migration, encryption, compression and data deduplication. This intelligence may be characterized as
- "back-end facing" intelligence; functionality that the facility's users don't get (direct) access to, only the administrators and back-end systems;
- "front-end facing" intelligence; functionality that the facility's users can access and make use of.

Note that File Storage is always implemented on top of some form of Raw Storage. As a consequence, a Pattern Variant dealing with file storage is often based on both the File Storage Pattern Type and the Raw Storage Pattern Type.

As examples: realizations of File Storage (and Raw Storage under it) can include:

Content Management Systems (CMS) and alike solutions,
(the front-end of) NAS appliances,
file share servers.

Note: it's possible that the solution is to make use of Business Rules (e.g. for data deduplication or automatic migration of data to nearline/offline storage). If this is the case, then this pattern must be paired with facilities that deliver this functionality.

Graphical Overview

This is the graphical representation of the infrastructure functions in this Pattern Type, plus their main relations:

(The source file of this picture can be downloaded here).

Pattern Type Composition

This pattern has the following mandatory and optional subfunctions, expressed in Building Block Types:

Icon	Function	WA	Inclusion	Rationale
	File Engine	SE	mandatory	This facility handles the actual storing/retrieving of loosely structured data such as (but not restricted to) files or documents. It serves end-users and systems (amongst which application engines and structured data stores). To that end, this facility offers a storage structure that fits the data to be stored, such as a file system for files, and an interface over which data can be offered or retrieved. The File Engine provides "intelligence" to the file storage solution with respect to its handling of the stored data, with the aim of meeting the quality requirements, improving infrastructure management, and/or providing sufficient performance. It may handle migration/replication to other instances of itself, encryption, compression, de-duplication and other "standard" storage-related processing. It also can handle granular, file-based permissions. Note that it requires Raw Storage to physically store the bits and bytes that make up the pieces of loosely structured data.
	Caching	MW	optional	An intelligent storage facility may use caching to increase its capacity to receive and/or return data on request.
	Control Interface	SE	optional	If implemented, this function can be used for several Management Tasks, among others: Restoring (partial) back-ups; Capacity & Performance Management; Provisioning; Rules Management, with which the stored data can be manipulated using metadata, usage, usage patterns or other information that the facility may obtain. Care must be taken to limit access to these management tasks to authorized systems and users.
	Restore	ST	optional	Implementing a facility of this type may serve different goals: It can serve a business need for data protection, It can support a technical need related to business continuity, technical exploitation or maintenance. To support these goals, Restore can support the following activities: restoring backups, either in entirety or partially; storage migration, where data that has been migrated off-line to backup media can be restored to active storage upon demand and/or rules. Note that the Restore function may or may not be under control of end users. In realization, Restore can be a sub-function of a versatile File Engine, e.g. the realized facility may be able to restore data from the file storage space under control of that File Engine, or it can be paired with a separate Pattern Variant of Raw Storage or File Storage/Raw Storage explicitly meant for backup & restore.
	Backup	ST	optional	Implementing a facility of this type always serves to support the Restore function, and thus virtually always appears together with one or more Variants of the Restore function. Furthermore, the criteria that govern the Backup functionality must always be derived from the goals and needs of all of the Restore Variants that it supports. Note that the Backup function usually is under control of IT Operations, but may occasionally be offered to end users directly. In its implementation, just as with Restore, Backup can be a sub-function of a versatile File Engine, so that the backups are created on the file storage space under control of that File Engine, or it can be paired with a separate Pattern Variant of Raw Storage or File Storage/Raw Storage explicitly meant for backup & restore.
	Archiving	ST	optional	Implementing a facility of this type in a File Storage Pattern Variant may serve different goals to have data immutably preserved for a certain period of time; to free up resources in the File Storage realization. To support these goals, Archiving offers to perform the following activities: to copy or move data to a secondary location, which is another (logical) location in this same File Storage realization, or another File Storage realization; and to do so according to specific rules and/or schedules; and in case of resource-saving archiving: to copy or move data back from the secondary location to the primary location according to demand and/or schedules; When the data has been created in the secondary location ("the archive"), it is accessible via a File Engine instance, often one that's either explicitly implemented for the archive.
	Scheduling	MW	optional	If the Pattern is to have automated backup and/or archiving functionality, then this function will govern the respective schedules and initiate separate backup and/or archiving jobs.

Pattern Type Neighbors

This pattern has the following mandatory and optional relations with adjacent (sub)functions, expressed in Pattern Types (PAT). Note: if the table below is empty, then there are no architecturally prescribed relations with adjacent subfunctions:

Function	Adjacency	Description
Raw Storage	mandatory	All "loosely structured data" eventually will have to be stored on some physical device, where it's "raw bits". Putting this Pattern under the File Storage pattern delivers this ability.
Authentication & Authorization	optional	A&A will be required for one or more of the following uses: access to the control interface, that may control aspects ranging from facility management to self-service provisioning of storage space; permission validation for read/write/modification/deletion actions on the loosely structured data being stored; permission validation for actions such as backup, restore and archiving.
Application Hosting	optional	One of the facilities that require use of file storage is the Application Hosting facility. An Application Hosting facility requires File Storage to store the applications themselves, and possibly the data with which the applications interact.

Pattern Variants based on this type

Pattern Variant	Brief description	Owner	maturity
PAV.File Storage.Local	File Storage.Local	J.A.H. Schoonderbeek	1