Cloud Explorer is a powerful GUI and CLI Amazon S3 client. In this release there is many code improvements to help users sync their data to a S3 bucket and migrate data between different S3 providers. There is also better support for S3 compatible servers such as Scality S3 Server and Minio.
Syncing, bucket migrations, and snapshots were completely rewritten for optimal performance. Now five sync tasks can run at the same time. Each task will check the file metadata and perform the necessary upload and download operations concurrently instead of a single operation at a time.
The Background sync feature enables users to perform a bidirectional sync on a folder like Dropbox in the GUI or CLI every five minutes. This feature was also rewritten and takes advantage of the improved syncing algorithms discussed earlier. Since it now runs in it’s own thread with a separate configuration file, users can use Cloud Explorer while the sync tasks run in the background.
Path Style access is now enabled for non-aws accounts providing better support for private S3 compatible servers like Scality and Minio. Users will now also be able to connect to these servers by IP address or DNS.
Regions have been removed from the code and configuration file. Cloud Explorer will retrieve the appropriate region from the S3 account resulting in better functionality and easier use. This being said, that means that previous Cloud Explorer configuration files will not work in the new release and the accounts will have to be added again.
The CLI now supports bucket snapshots and migrations with the ability to use environment variables instead of a configuration file. This functionality makes it easier to run in a container such as Docker or Rocket.
I hope that you will enjoy this exciting new release and please provide feedback on the GitHub page or directly to me on Twitter.
In my previous blog posts, I always talked about my setup but never showed a completed diagram. In the above image, you can see everything that I am running on my 6 node Raspberry Pi cluster. All of the applications above are running in Docker containers managed by PiCluster except for Gluster.
I did some updating to Cloud Explorer recently to make it work with Minio. Minio is one of many open source S3 servers available today for people to use on-premises for their personal cloud storage needs. With the added support, Minio users can take advantage of Cloud Explorer’s unique features such as performance testing, note taking, playing music, viewing images, and search.
My previous post was about using Cloud Explorer with the Scality S3 server. After I published that post, I thought it would be informative to go one step further and explain how I use the S3 server in homeduction (applications run at home in production). My homeduction environment consists of four Raspberry Pi’s running Docker that power this WordPress blog and many other applications . My goal is to add an S3 server to store the images for this blog and anything else that I can come up with.
I spent a few weeks searching for an open-source S3 server that I can run at home to test Cloud Explorer. I first came across Minio which is an open-source S3 server but I could not get it to work with Cloud Explorer because it had issues resolving bucket names via DNS which is a requirement using the AWS SDK. I then read an article about Scality releasing an open-source S3 server that you can run inside a Docker image. I was able to get Scality up and running quickly with little effort. In this post, I will explain how I got the Scality S3 server setup and how to use it with Cloud Explorer.
Cloud Explorer is a open-source Amazon S3 client that works on any operating system. The program features a graphical or command line interface. Today I just released version 7.2 and hope that you give it a test drive. Feedback and uses cases are always encouraged.
What’s new in this release?
To start, this release of Cloud Explorer was compiled with Java 1.8.0_72 and version 1.10.56 of the Amazon S3 Java Development Kit ( JDK). The major improvements in this release regard file synchronization. Basically, it was mostly rewritten. By putting forth the effort, it helped reduce technical debt and consistency between the command line and graphical version of Cloud Explorer.
How do I get it?
Cloud Explorer v7.2 is available under the “Downloads” section of the Release page on GitHub. Simply click on “cloudExplorer-7.2.zip” and the download will begin. When the download is finished, extract the zip file and double click on “CloudExplorer.jar”.
Where do we go from here?
I know it has been a while since Cloud Explorer has been touched. It is hard to handle a project all by yourself and keep innovating. I feel that with this release, Cloud Explorer reached a stable point. I am always looking for new ideas and help from the community. If you are interested in contributing, please contact me or open an issue on the GitHub page.
I have been working hard on new features and GUI enhancements for Cloud Explorer 6.0. I think it is important for an application to improve over time and this project has seen it’s share of visual evolutions. If you look back at the 1.0 release, the changes are night and day. I enjoy the challenge of making this program look pretty for each major release and hope that the users find it to be good. This release uses many free-licenses images and GPL images found in the KDE Desktop Environment.
If you like to test this early release, please create an issue on GitHub and I will provide a release candidate. I can always can use help in testing. You can also build the latest version of CloudExplorer yourself. The expected release date for 6.0 is July 22, 2015.
1. Fixed a bug when switching accounts.
2. If no region is specified, it will default to “defaultAWS” to
avoid a null pointer exception. To resolve the issue, delete and
add your account back.
3. If no image is selected when the image button is pressed, nothing
4. If no music is selected when the play button is pressed, nothing
5. Support for line spaces in s3.config
6. Versioning display fix.
1. Snapshots and Migration are now located under the new menu
“Snapshots and Migration”.
2. New Icons for GUI.
3. Maximize Window.
4. For improved GUI responsiveness, many actions now avoid
1. Create Bucket snapshots.
2. Restore Bucket snapshots.
I am pleased to announce v5.6 of Cloud Explorer. The biggest new feature for this release allows users to record audio messages that will be saved into their selected bucket. Cloud Explorer provides a great way to share audio messages because the S3 API allows users to share files via a public URL. Syncing is now more stable intelligent. When syncing to or from a bucket, sync compares the last modified time and if the destination has a newer version, the file will not be copied. The text editor now has a folder list box so users can save notes into a specific folder. This is an excellent way to stay organized and use Cloud Explorer for note taking.
Getting Cloud Explorer
Cloud Explorer is available for download from here. After downloading, please upgrade to 5.6 if you are running an earlier release.
Upgrading to 5.6
Starting with Cloud Explorer 5.0 or later, you can upgrade by clicking on Help -> Check for updates. After the update is complete, restart Cloud Explorer.
Here is a complete list of changes in v5.6:
1. Syncing GUI and CLI: Prevent duplicated transfers of synced files in folders and items go in the correct folder when syncing to the local machine,
2. Syncing from CLI: Syncing from S3 saves to appropriate directory level.
1. Music Player: Plays WAV files, stop button renamed to “Stop /Close” for clarity, and no longer case sensitive for MP3 file extensions.
2. Syncing GUI and CLI: Overwrite button removed.
1. Audio recorder.
2. Sync GUI and CLI: Timestamps are compared and the outdated files are overwritten.
3. Folder support for saving files in the Text Editor.
I pondered hard to think how I can make the build process easier for Cloud Explorer by using Cloud Explorer. Currently, I have a bash script that runs and puts files into place locally and compresses the program directory into a zip file. Finally, I have to manually upload Cloud Explorer to the S3 account for sharing. This process involves multiple steps and is tedious to do from different locations. For example, If I want to build a copy at another location, I would have to manually copy the file to my location and then upload with a client. There has to be an easier way.
For a more efficient solution, I added a command line argument for Cloud Explorer that will upload a given file with an object name by my choosing to a specified bucket. After the upload is completed, permissions will be set automatically and configured for sharing.
The above command is contained in a bash script that runs Cloud Explorer to do the upload. The build argument followed by the remaining arguments runs the program in “Build Mode”. The $BUILD_NAME argument specifies the name of the file when stored on S3. The $ZIP argument contains the location of the Cloud Explorer zip file. The $BUCKET argument specifies the bucket to be used on the S3 server. The account used for the upload will be the first account listed in the ~/s3.config file.
When the build system is ran and the arguments are accepted, Cloud Explorer will perform a parallel multi-part upload of the zip file. Upon completion, the zip file will have public access and a signed URL for simplified sharing. The signed URL will be displayed in the terminal and I can copy and paste it to my peers to download.
By adding this support to Cloud Explorer, I can run my build script and then wait a few minutes to share the build with anybody.