From Pydio Android to Pydio Pro: journey of a full rewriting.

Although already powerful and having received a very good rating on the Google Play Store, our previous Pydio Android application was lacking some much awaited features. But the underlying code, that was very out of date (it dated back to the first release of the android app), hit some severe limitations in terms of architecture. SDK was very hard to maintain, UX application code and SDK had too many dependencies, etc… We first redesigned the Java SDK to make it more modular, ready to communicate with the new REST API. Then at one point, rewriting the Android application based on the new SDK was simpler than just overhauling the existing code, It gave us the possibility to implement many new features, but also dust off the UX.

Pydio Android Application: in need of some an overhaul

The handling of the Android application

When I started as principal android developer at Pydio, the Android application was already in production with a rating of four stars and was fairly stable. I spent my first weeks fixing reported bugs and improving the GUI. This was also a way for me to familiarize myself with the application and all the code that composed it.

A few months later, with a growing list of requested features I started to make important improvements in the application. This was the beginning of a long journey.

Lots of coding for even the simplest improvements

The application was largely improved in terms of functionalities and UI but the code reached a point where it was hard to maintain. Feature after feature the code was getting cumbersome and at the same time the most requested features were not all implemented. The reason is the SDK was not designed to handle the advanced features we wanted. Adding a new feature to the application was a very painful task as it often lead to breaking the architecture rules. From that point it appeared obvious to us that rewriting the SDK was the urgent thing to do. So I got to work right away to write a new SDK from scratch.

Creation of the Pydio Pro application

Of course the objective was not to fully rewrite a new SDK but to redesign the architecture so that we have an easily maintainable layered SDK with only necessary external dependencies. So the new SDK reused as much of the old SDK as possible. It did not take  long to have a basic working version of the new SDK so I started testing in order to replace the old SDK in the android application. Introducing the new SDK in the application appeared to be a very difficult operation. Indeed it implied a lot of changes in the application to make it run over the new SDK. Clearly, taking into account the new SDK architecture and all the possibilities it gave us, creating a new Android application based on the new SDK became the obvious choice over overhauling the old one and trying to get it to use the new SDK.

The decision to write a new Android application was made, and the expectations were high. Needless to say that the new application, that we eventually decided to call Pydio Pro, had to be 10 times better than the old one. In addition to the existing features in the previous Android application, the Pydio Pro application had to absolutely provide all the important requested advanced features and all in a full material design like all modern Android applications. The design was a key point for improvement as its was one of the biggest weaknesses of the old application. We raised the expectations even higher as we wanted to make the new application look the same on every android device regardless the Android OS version.

Welcome Material Design

After months of intense development, we published The Pydio Pro Android application in beta. As requested, we included the old application features and added many of the requested features. Beta testers have reported being happy with the new UI changes. This has been achieved by implementing the material specifications as defined by Google. The specifications explains in detail how to build full material UI components

Material design specs

Furthermore some of the default UI components provided by the Android library like TextInput and Button had to be customized to look as specified in the material design specifications but requires them to appear the same regardless the device Os version.

Reflecting server state in real time

In addition to the new design and unlike the old application, the new application is now more responsive to the server's configuration. Indeed configuration changes on server side are reflected on the application.

All Pydio Servers store their settings in an XML Document called the “registry”. This registry is updated each time the server settings changes. Basically the registry tells what are workspaces access rights and which features are enabled and which function can be called. The Pydio Pro application periodically asks for configured servers registry in order to have the latest settings on application side. That way when building the menus and the file options list only features and operation that are enabled in the registry are made available.

Offline sync & backup

In addition to the increased reactivity, the Pydio Pro application has some interesting features like the Offline and Backup.

Offline sync feature

The most requested is the offline sync that allows user to keep files on their device even when servers are not reachable. The offline sync could not be added without the new Changes API on Pydio Servers.

The Changes API have been first introduced as a plugin on Pydio Server and was designed to work with Pydio Sync Desktop, our file synchronizing desktop application. I took part of the development of this application prior to start the Pydio Pro application and I can tell that the concept behind the Changes API is very clever.  It provides the most efficient methods to synchronize files when working on a Pydio server.

Basically the API is able to trace the sequence of changes that occurred on files since the moment they are created. This is what a Changes API response fragment looks like :

{

"Seq":115662,

"Node_id":40990,

"Type":"create",

"source":"NULL",

"target":"\/Test.pdf",

"Node":

               {

               "Bytesize":22068,

               "Md5":"afe0272bb811b26cec7ad4a2e19f86f3",

               "Mtime":1457718231,

               "node_path":"\/Test.pdf",

               "Repository_identifier":"1-jabar"

               }

}

Here we requested sequence of changes that occurred in a certain folder and the response tells that only a file with name Test.pdf has been created in the folder.

With that changes mechanism synchronizing folders was very easy to implement. In the Pydio Pro I simply periodically get changes and process them instead of comparing and process file trees content diff. The tricky part about the Offline sync implementation was to find the best way to repeatedly request the API without reducing the whole application autonomy. To accomplish that I used some useful classes provided by the Android API like  PendingIntent, Services and AlarmManager to run a background service that efficiently schedule requests.

Backup feature

The Backup feature  allows users to create a restore point of files present on their mobiles. Basically the user selects folders on their device, configures the server where files have to be stored and finally chooses how frequent backup has to be done.

Since the Backup process runs between a device to a Pydio server there is no provided API like in Offline sync. Instead the Backup takes advantage of the application folder tree cache in order not to upload files that are already on the server. On the other hand the Backup uses the same scheduling mechanism as the Offline sync to maintain the application ergonomy.

Conclusion

Today the Pydio Pro application is still in beta and is tested by volunteers to improve it and make it stable. There are features that have not been added yet in development and their publication will be announced after the official release of the Pydio Pro application.

Author : Jabar KARIM