Manage Buckets and Files in BucketFS
This article explains how to access and manage buckets and files in BucketFS.
You can access a BucketFS bucket and the files in the bucket from outside the cluster by using an HTTPS client such as curl. To do this, you need to provide the IP address of one of the database nodes, the BucketFS port, the bucket name, and the read/write passwords.
The read and write passwords can be set when you create the bucket. For more details, see Create New Bucket.
To configure passwords for an existing bucket, use the ConfD job bucket_modify.
If the bucket is not publicly readable, you can only access it using the configured read/write passwords.
The write password allows both read and write access.
The following examples show how to use curl to list the existing buckets in a BucketFS service (defined by its IP address and port) and how to manage the buckets and files. The relevant parameters used in these examples are:
| Parameter | Value in the examples |
|---|---|
| Database node IP address | 192.168.6.75 |
| BucketFS port | 1234 |
| Bucket name | bucket1 |
Read password (r) |
readpw |
Write password (w) |
writepw |
Using curl
Curl is a command-line tool for transferring data using URLs. For more details about how to use curl, refer to the curl manpage.
Request method
You can specify the HTTP request method used by curl explicitly by using -X [METHOD]. For example: curl -X PUT my_file http://example.com/my_file. If you do not specify a HTTP request method, curl will choose a default method based on other parts of the command. For example, sending just curl <url> without any options will default to the GET request method, while curl -T <filename> <url> is the same as using curl -X PUT <filename> <url>. The -X prefix is required in some cases, such as when using the DELETE request.
Show fail info in output
If you include the -f option curl will output server errors, which can be helpful when debugging your scripts. Without the -f option the connection will fail silently in case of a server error. In the following example, curl is passing a HTTP URL but the server requires HTTPS, which is indicated by the curl output:
curl http://r:readpw@192.168.6.75:1234 -f
curl failed to verify the legitimacy of the server and therefore could not
establish a secure connection to it.Skip TLS certificate check
If you include the --insecure or -k option, curl will not verify that the server’s TLS certificate is valid before proceeding. For example:
The option --insecure or -k tells curl to bypass verification of the security of the connection. Only use this option if certificate validation is not possible and you trust the server.
List buckets
To list all buckets on a BucketFS service, use curl with the IP address of the database node and the BucketFS service port:
In this example, the BucketFS service 192.168.6.75:1234 contains the buckets bfsdefault and bucket1:
Add files to a bucket
To add files to a bucket, use curl -T or curl -X PUT:
curl -T <path-to-local-file> https://w:<write-password>@<ip>:<port>/<bucket-name>/<path-to-bucket-file> [-f] [--insecure]If the upload path does not include a target filename, curl will store the file in BucketFS using the original filename.
If the filename for the target file is omitted, the path to the bucket must end with a trailing / (forward slash).
For example, to upload the files file1 and
tar1.tgz into the bucket bucket1:
curl -T file1 https://w:writepw@192.168.6.75:1234/bucket1/
curl -T tar1.tgz https://w:writepw@192.168.6.75:1234/bucket1/To store the file tar1.tgz as my_file.tgz:
View files in a bucket
To view a list of the files in a bucket, use curl with the read or write password and the path to the bucket:
For example:
To view the contents of a specific file in a bucket, use curl with the read or write password and the path to the file:
For example:
Delete files from a bucket
To delete files from a bucket, use the DELETE request method and the write password:
curl -X DELETE https://w:<write-password>@<ip>:<port>/<bucket-name>/<path-to-bucket-file> [-f] [--insecure]For example, to delete the file file1 from the bucket bucket1:
