Cloud Explorer is a powerful GUI and CLI Amazon S3 client. In this release there is many code improvements to help users sync their data to a S3 bucket and migrate data between different S3 providers. There is also better support for S3 compatible servers such as Scality S3 Server and Minio.
Syncing, bucket migrations, and snapshots were completely rewritten for optimal performance. Now five sync tasks can run at the same time. Each task will check the file metadata and perform the necessary upload and download operations concurrently instead of a single operation at a time.
The Background sync feature enables users to perform a bidirectional sync on a folder like Dropbox in the GUI or CLI every five minutes. This feature was also rewritten and takes advantage of the improved syncing algorithms discussed earlier. Since it now runs in it’s own thread with a separate configuration file, users can use Cloud Explorer while the sync tasks run in the background.
Path Style access is now enabled for non-aws accounts providing better support for private S3 compatible servers like Scality and Minio. Users will now also be able to connect to these servers by IP address or DNS.
Regions have been removed from the code and configuration file. Cloud Explorer will retrieve the appropriate region from the S3 account resulting in better functionality and easier use. This being said, that means that previous Cloud Explorer configuration files will not work in the new release and the accounts will have to be added again.
The CLI now supports bucket snapshots and migrations with the ability to use environment variables instead of a configuration file. This functionality makes it easier to run in a container such as Docker or Rocket.
I hope that you will enjoy this exciting new release and please provide feedback on the GitHub page or directly to me on Twitter.
In my previous blog posts, I always talked about my setup but never showed a completed diagram. In the above image, you can see everything that I am running on my 6 node Raspberry Pi cluster. All of the applications above are running in Docker containers managed by PiCluster except for Gluster.
I am pleased to announce PiCluster v1.7. In this release, I wanted to make PiCluster easier to use by having the Web Console handle most of the common configuration file changes. Not everyone enjoys editing json files including myself. Now let’s go over what is new in this release.
I did some updating to Cloud Explorer recently to make it work with Minio. Minio is one of many open source S3 servers available today for people to use on-premises for their personal cloud storage needs. With the added support, Minio users can take advantage of Cloud Explorer’s unique features such as performance testing, note taking, playing music, viewing images, and search.
I am pleased to announce v1.6 of PiCluster. In this release there are a few usability bugs fixed and a new feature that allows you to change the host of a running container. Having the ability to easily change where a container is running is a standard and crucial feature to expect from a container management platform. I am glad that it is finally here and let’s explore how it works!
I am pleased to announce the new version of PiCluster. In this release, users can connect to a host running an rsyslog server and the PiCluster agent to view the log drain in the PiCluster web console and run searches. This combined integration provides a single pane of glass to monitor physical hosts and Docker containers easily. Let’s take a look on how to enable this functionality.
I switched to Gluster FS a while ago to provide easier container mobility across my Raspberry Pi Docker Cluster. Gluster worked great and was easy to get up and running but I had very poor performance. The average write speed was about 1 MB/s which is unacceptable for a filesystem that will undergo a lot of writes. I decided that it was time to take action and started looking at kernel parameters that could be changed.
I spent a few weeks searching for an open-source S3 server that I can run at home to test Cloud Explorer. I first came across Minio which is an open-source S3 server but I could not get it to work with Cloud Explorer because it had issues resolving bucket names via DNS which is a requirement using the AWS SDK. I then read an article about Scality releasing an open-source S3 server that you can run inside a Docker image. I was able to get Scality up and running quickly with little effort. In this post, I will explain how I got the Scality S3 server setup and how to use it with Cloud Explorer.
I have been a hardcore CentOS user for many years now. I enjoyed its minimal install to create a light environment, intuitive installation process, and it’s package manager. Docker is the most popular container format today and provides developers and enthusiasts with an easy way to run workloads in containerized environments. I started using Docker in production at home for about a year now for services such as Plex Media Server, Web Server for this blog, ZNC, MineCraft, and MySQL to name a few. A Dockerfile is a set of instructions used to create a Docker image. I invested many hours creating perfect Dockerfiles using CentOS and Fedora to make deployments simple on any operating system. However, a personal revolution was brewing.