GP.Managed File Transfer

	GP	Managed File Transfer	Version:	0.2
	Document type:	Generic Pattern	Owner:	J.A.H. Schoonderbeek

Description

This Generic Pattern belongs to "Business Support". Managed File Transfer (abbreviated MFT) indicates the automated transfer of data in a loosely structured form ("files") between IT facilities. These facilities can be owned by the organization, but an MFT instance may also be used to transfer files into or out of the organization, from or to external IT systems. Furthermore, human users may also make use of the MFT pattern to exchange data with other users and/or IT systems. A realization of this MFT Pattern handles the reception and/or collection of files from a source, and their delivery to and/or collection by one or more targets. It may also additionally modify the files by transcoding, encrypting/decrypting, (de)compressing, and/or it may provide means to modify the files in transit in other respects. Incoming and outgoing traffic may be regulated using predescribed rules and/or by schedules that act on times, periods or order. The Pattern requires use of an instance of Data Transport (and optionally Message Handling), since the data to be transported comes in and goes out over these services. Most data transfers will require the Managed File Transfer Pattern to authenticate itself to the source and/or target. Furthermore functionality can be present to provide logging of each file transfer for operational and security purposes.

Services realized

This Pattern realizes the following service(s):

Managed File Transfer (This service provides a means to transport loosely structured data from a source to one or more targets, either ad hoc or structurally.)

Functional and Integration view

This is the graphic representation of the functional model of this Generic Pattern:

Generic Pattern Composition

This pattern is an aggregation of the following (mandatory and optional) functions, expressed in Generic Functions:

Icon	Function	Inclusion	Rationale
	File Engine	recommended	This function represents the core intelligence of the Managed File Transfer service. This function performs the collection or accepting of files from the source, and the transmission or offering of files to one or more targets.
	Presentation Engine	recommended	This function represents the transcoding that is required when a source offers a file in a different format and/or using a different transfer protocol from the format/protocol used by the file transfer's target(s). Furthermore, if the Pattern offers ad hoc file transfers to human users, then this function shapes the users' experience when using the Managed File Transfer service (e.g. a web environment for uploading and downloading the files).
	Caching	recommended	Any file in transit will be buffered in the Managed File Transfer facility while it is processing file transfer tasks such as transcoding. This transient storage of files inside the MFT facility needs attention because of three aspects: Security: the storing of files needs to be secured to the level that's appropriate for the confidentiality required by the business and security department; Storage space: enough space needs to be present to store the maximum volume of files that may reside in transient storage at the facility at any given time; an overflow to the adjacent service File Storage may be in order; Business continuity: when a technical problem or disaster impedes the MFT facility, measures must be taken to account for files that were residing only in transient storage at the time of the problem.
	Rules Engine	optional	This function represents the additional intelligence that can drive extra manipulation of files in transit, advanced exception handling, logging and/or routing by the Pattern. Active collection of information on the files in transit, source and/or target(s) may lead to execution of (implicit or explicit) rules.
	Logging	optional	This function is often included, as logs of file transfers are required not only by Operations, but also Security/Compliance. Furthermore, a business entity responsible for sending files, or one that expects incoming files, may require access to the logged information to verify data transfers.
	Scheduling	optional	Scheduling can serve to initiate file tranfers and/or internal maintenance tasks, on regular or predetermined times.
	Controlling	recommended	This function models the manner in which the Pattern's administrators, and possibly authorized clients, can administer file transfers. Among the administrative tasks can be the following: Administering file transfer end points; Administeringfile transfer tasks (which in turn make use of transfer end points); Administering schedules for automated file transfers. Since the files transferred by the Pattern may be business critical and/or have a certain level of confidentiality, care must be taken to limit access to this function to authorized users.
	Encryption	optional	This function can be used to model the encryption/decryption that is associated with secure file transfer protocols, as well as the manipulation of files in transit, when the MFT is required to encrypt/decrypt files or file archives.
	Reduction	optional	This function can be used to model the compression that is often associated with file transfer protocols, as well as the manipulation of files in transit, when the MFT is required to compress/decompress files or file archives, or manipulate files that sit inside a file archive.
	Distribution	optional	When the MFT facility is required to deliver files to multiple targets, then this function models the handling of addressing and routing.
	Message Formatting	optional	When the MFT facility makes use of a Message Handling services, then the Message Formatting functionality provides the interface to such a service.

Services connected with this Generic Pattern

This Generic Pattern has the following mandatory and optional relations with adjacent Generic Services.

Service	Adjacency	Summary	Rationale
Data Transport	recommended	This service transports data between automated systems.	All files that an MFT facility handles come in and go out over a Data Transport service. Especially the transfer of files over one or more security delineations require attention, so the explicit modelling of Data Transport is recommended.
Authentication & Authorization	optional	This service can validate an identity claim, and it can validate the permissions required for an action, as part of an Authentication & Authorization process.	The Pattern will likely be required to authenticate itself when collecting and/or delivering a file. Furthermore, the Pattern probably will require systems and users to authenticate themselves when they wish to transfer or collect files. Finally, access to the services provided by the Managed File Transfer Pattern must likely be limited to authorized personnel.
Facilities Monitoring	optional	This service allows its users to monitor IT facilities with the aim of guarding operational continuity or security.	The logs of file transfers are of interest to security and operational personnel, as well as the business units that make use of Managed File Transfer. Connections with Facilities Monitoring services provide these monitoring services with the data with which they can provide their respective users the required insights and overview.
File Storage	recommended	This service offers clients the ability to store, retrieve and modify data in loosely structured form.	A file in transit will be stored in the Managed File Transfer facility when the target is required to actively fetch the file ("pull-out") instead of having the file delivered as soon as it enters the MFT from the source. This storage of files inside the MFT facility needs attention because of three aspects: Security: the storing of files needs to be secured to the level that's appropriate for the confidentiality required by the business and security department; Storage space: enough space needs to be present to store the maximum volume of files that may reside at the facility at any given time (including exceptional times such as when one or more sources cannot accept files for any given reason, e.g. connectivity issues); Business continuity: when a technical problem or disaster impedes the MFT facility, files already accepted from the source but not yet delivered to (all) targets may be at risk of loss or corruption. The connection between the MFT facility and the File Storage service should correctly handle such events.
Data Protection Management	optional	This service provides the ability to protect data against accidental or wilful corruption.	When data en route between sender and recipient(s) is residing in a Managed File Transfer facility, it needs to be protected from corruption and data loss, possibly even in the event of a total loss of the MFT facility itself. Adequate protection may be obtained by calling upon an external Data Protection Management service.
Message Handling	optional	This service handles transport, storage and delivery of messages between senders and recipients.	This service may be adjacent because of either of the following reasons: files are transported in and/or out of the MFT facility via a messaging protocol; the MFT facility uses Message Handling services to notify security, operational and business users about events in the facility, such as failure/success of file transfers.

Applied Patterns based on this Generic Pattern

The following Applied Patterns are based wholly or in part on this Generic Pattern: