Manage Buckets and Files in BucketFS

This article explains how to access and manage buckets and files in BucketFS.

You can access a BucketFS bucket and the files in the bucket from outside the cluster by using an HTTPS client such as curl. To do this, you need to provide the IP address of one of the database nodes, the BucketFS port, the bucket name, and the read/write passwords. You may also have to adjust your internal firewall configuration to allow the connection.

The read and write passwords can be set when you create the bucket. For more details, see Create New Bucket.

If the bucket is not publicly readable, you can only access it using the configured read/write passwords.

The write password allows both read and write access.

The following examples show how to use curl to list the existing buckets in a BucketFS service (defined by its IP address and port) and how to manage the buckets and files. The relevant parameters used in these examples are:

Parameter Value in the examples
Database node IP address 192.168.6.75
BucketFS port 1234
Bucket name bucket1
Read password (r) readpw
Write password (w) writepw

Using curl

Curl is a command-line tool for transferring data using URLs. For more details about how to use curl, refer to the curl manpage.

Request method

You can specify the HTTP request method used by curl explicitly by using -X [METHOD]. For example: curl -X PUT my_file http://example.com/my_file. If you do not specify a HTTP request method, curl will choose a default method based on other parts of the command. For example, sending just curl <url> without any options will default to the GET request method, while curl -T <filename> <url> is the same as using curl -X PUT <filename> <url>. The -X prefix is required in some cases, such as when using the DELETE request.

Show fail info in output

If you include the -f option curl will output server errors, which can be helpful when debugging your scripts. Without the -f option the connection will fail silently in case of a server error. In the following example, curl is passing a HTTP URL but the server requires HTTPS, which is indicated by the curl output:

curl http://r:readpw@192.168.6.75:1234 -f

curl failed to verify the legitimacy of the server and therefore could not
establish a secure connection to it.

Skip TLS certificate check

If you include the --insecure or -k option, curl will not verify that the server’s TLS certificate is valid before proceeding. For example:

curl https://r:readpw@192.168.6.75:1234 -f --insecure

The option --insecure or -k tells curl to bypass verification of the security of the connection. Only use this option if certificate validation is not possible and you trust the server.

List buckets

To list all buckets on a BucketFS service, use curl with the IP address of the database node and the BucketFS service port:

curl https://<ip>:<port>

In this example, the BucketFS service 192.168.6.75:1234 contains the buckets bfsdefault and bucket1:

curl https://r:readpw@192.168.6.75:1234
bfsdefault
bucket1

Add files to a bucket

To add files to a bucket, use curl -T or curl -X PUT:

curl -T <path-to-local-file> https://w:<write-password>@<ip>:<port>/<bucket-name>/<path-to-bucket-file> [-f] [--insecure]

If the upload path does not include a target filename, curl will store the file in BucketFS using the original filename.

If the filename for the target file is omitted, the path to the bucket must end with a trailing / (forward slash).

For example, to upload the files file1 and tar1.tgz into the bucket bucket1:

curl -T file1 https://w:writepw@192.168.6.75:1234/bucket1/
curl -T tar1.tgz https://w:writepw@192.168.6.75:1234/bucket1/

To store the file tar1.tgz as my_file.tgz:

curl -T tar1.tgz https://w:writepw@192.168.6.75:1234/bucket1/my_file.tgz

View files in a bucket

To view a list of the files in a bucket, use curl with the read or write password and the path to the bucket:

curl https://r:<read-password>@<ip>:<port>/<bucket-name> [-f] [--insecure]

For example:

curl https://r:readpw@192.168.6.75:1234/bucket1
file1
tar1.tgz
my_file.tgz

To view the contents of a specific file in a bucket, use curl with the read or write password and the path to the file:

curl https://r:<read-password>@<ip>:<port>/<bucket-name>/<path-to-bucket-file> [-f] [--insecure]

For example:

curl https://r:readpw@192.168.6.75:1234/bucket1/my_folder/file1

Delete files from a bucket

To delete files from a bucket, use the DELETE request method and the write password:

curl -X DELETE https://w:<write-password>@<ip>:<port>/<bucket-name>/<path-to-bucket-file> [-f] [--insecure]

For example, to delete the file file1 from the bucket bucket1:

curl -X DELETE https://w:writepw@192.168.6.75:1234/bucket1/file1