Pydio Cells & Enterprise 3.0.0

Cells V3 provides a new data source format that dramatically improves access speed. We have rethought our microservices deep in the core of the code to help streamline security implementation, supercharge searchability via new metadata, and make compliance monitoring and reporting easier.

New Datasource Format

An often-asked Pydio user requirement was to "keep the tree structure" of files visible on the storage (and modify these directly without going through Pydio). To achieve this, Cells datasource relies on a unidirectional synchronization between the storage and the internal index. It brings its share of issues and performance limitations when the number of files is getting high, and sometimes a "resync" of the datasource is required to fix index issues.

Introducing a new datasource format, that keeps the tree structure only in Cells internal indexes, and stores files as a flat structure on the storage. This is more in line with the "object storage" design, and brings huge performance gains as no more "sync" is required to maintain the indexes. This is now the preferred format unless direct access to the files on the storage is necessary. The ability to recover the tree structure is guaranteed by tools providing import/export of the structure on-file.

This new format is now applied to "internal" buckets previously used for storing thumbnails and versions, thus providing more flexibility, as they now appear as standard datasources (see Versioning below). Finally, S3 (and compatible) support is improved, and Cells S3 gateway (exposing Cells files as an S3-compatible storage) was made more vanilla with the support of listBuckets request and better Content-Type extraction.

UX

Tons of bug fixes and small improvements were done in all areas of the GUI (JavaScript is now built on top of React 17). Files presentation now offers a new "Gallery" view perfect for displaying folders of photos ("masonry layout").

Many new metadata types are available, with more meaningful fields for boolean, dates, integers, progress bars, etc... They are indexed and searchable and can be used in Cells Flows as well. The search interface has been fully redesigned for improved searchability, filling the whole screen and giving the ability to preview files from any workspace without leaving the search results view.

Cells Flows new features

Cells Flows jobs can now listen to many events, have dedicated conditional branches to handle them, and group similar jobs into one. For example, previous "thumbnails creation" and "thumbnails deletion" are now grouped into one job. In conjunction with the new metadata types, Cells Flows now allows the creation of advanced jobs such as Validation/Approval scenarios, including reminders, alerts, and file tagging.

Cells Convert Tools is a brand new Docker image that can be spawned on a dedicated server and used in conjunction with Cells Flows to generate thumbnails and PDF previews of most file formats. Check it out!

Security & ACLs

[ED] Authentication can now be hardened with DUO Security multi-factor service.
[ED] New security policies permissions can be applied to disable specific actions (DELETE, DOWNLOAD, UPLOAD) and fine-tune permissions.
Security fixes from the 2.2.X branch are reported.

Redesigned versioning

Now that the "versions" datasource uses the new flat storage standard, it is easier to shard versions on multiple locations, enable encryption, and many more. Versioning policies now provide a specific behavior for handling revisions after the original file is definitely deleted. By default, they are moved inside a dedicated folder in the versions' datasources, allowing administrators to easily find the last revision and restore it on behalf of a user.

Scalability / Provisioning

Performances are improved on many aspects, from the new datasource format to a redesign of the message bus that now does not require NATS anymore on a single-node deployment. The focus has been put on in-life maintenance of the system: housekeeping logs and activities, better log formats, caching, CPU and RAM consumption lowered.

Installation can be more easily automatized with the support of new fields in YAML/JSON configs to set multiple sites and custom configuration keys at startup.

Contributions

This release comes with a fully translated Português do Brasil version thanks to Claudio Ferreira (@filhocf).

If you want to help us and participate by adding the translation to your language, it is really easy: just navigate to the Pydio Cells project in Crowdin, create an account and get started!