Skip to main content

2021-19 Weekly Report

· 5 min read

Hello everyone, here is the weekly report for BeyondStorage, range from 2021-05-10 to 2021-05-14.

Library#

Path Style Support for go-service-qingstor#

vhost and path style are two different endpoint styles which are defined in QingStor:

  • Virtual-host Style: http://<bucket-name>.<zone-id>.qingstor.com/<object-name>
  • Path Style: http://<zone-id>.qingstor.com/<bucket-name>/<object-name>

For some reason, dns is not available for vhost, so we need to use path style to detect bucket location.

@bokket made his first commit to solve this by Use path style instead of vhost and service: Fix location not detected correctly. Great job!

For more details, please refer to QingStor detect use path style instead of vhost.

Idempotent Storager Delete Operation#

We use Delete to handle all object delete operations, but their behavior is not unified and well-defined.

After @Xuanwo made this proposal: GSP-46: Idempotent Storager Delete Operation, @JinnyYi is now finished all the implementation in different services except USS, which can not pass our integration tests, because USS requires a short interval between PUT and DELETE, or we will get this error: DELETE 429 {"msg":"concurrent put or delete","code":42900007,"id":"xxx"}.

For more details, please refer to Implement GSP-46: Idempotent Storager Delete Operation.

BeyondStorage Error Handling#

@xxchan has designed the new error handling proposal: GSP-47: Additional Error Specification last week.

In this week, to distinguish our errors more convenient, @xxchan also made the proposal: GSP-51: Distinguish Errors by IsAosError, which introduces the interface AosError:

type AosError interface {    // IsAosError SHOULD and SHOULD ONLY be implemented by error definitions in go-storage & go-service-*.    // We depends on the AosError interface to distinguish our errors.    // There's no need for user code to implement or use this function and interface.    IsAosError()}

For now, he has finished the implementation for both proposals above in different services, and the Error Handling Doc is also added for reference. Nicely done!

For more details, please refer to Implement GSP-47: Additional Error Specification.

Add object mode check for operations#

In GSP-25, we added support for object modes by bit map. All available object modes are listed below:

type ObjectMode uint32// All available object modeconst (    // ModeDir means this Object represents a dir which can be used to list with dir mode.    ModeDir ObjectMode = 1 << iota    // ModeRead means this Object can be used to read content.    ModeRead    // ModeLink means this Object is a link which targets to another Object.    ModeLink    // ModePart means this Object is a Multipart Object which can be used for multipart operations.    ModePart    // ModeBlock means this Object is a Block Object which can be used for block operations.    ModeBlock    // ModePage means this Object is a Page Object which can be used for random write with offset.    ModePage    // ModeAppend means this Object is a Append Object which can be used for append.    ModeAppend)

It is intended to check object mode at the start of specific operation. For instance, both WritePart and WriteAppend got a pointer to Object as an input, we need to ensure this Object is available for certain operation, so we should add object mode check and return ObjectModeInvalidError(introduced in GSP-47) asap if Object not fit.

So @Prnyself made a proposal: GSP-61: Add object mode check for operations to generate mode check, instead of user implementation in specific actions. All the implementation for different services are finished now.

For more details, please refer to Implement GSP-61 Add object mode check for operations.

WriteMultipart returns Part#

Multiparter is designed for multipart upload. Multipart upload is a three-step process:

  • CreateMultipart is used to initiate the upload.
  • WriteMultipart is used to upload the object parts. And ListMultipart could be used to list all of your in-progress multipart uploads or get a list of the parts that you have uploaded for a specific multipart upload.
  • CompleteMultipart is used to complete the multipart upload after you have uploaded all the parts.

CompleteMultipart request must include the upload ID and a list of both part numbers and corresponding ETag values returned after those parts were uploaded in some services. The ETag uniquely identifies the combined object data, not necessarily an MD5 hash of the object data. We need return ETag that we got from services to make it possible.

So @JinnyYi made a proposal: GSP-62: WriteMultipart Returns Part, which introduced a break change: return *Part in WriteMultipart, which should be held and passed into CompleteMultipart as a param.

This proposal's implementation has been finished by @JinnyYi in services implemented Multiparter, and the go-integration-tests and multiparter docs are also updated. Good Job!

For more details, please refer to Implement GSP-62: WriteMultipart returns Part.

Multipart upload part number check in go-service-qingstor#

This week, we have a new contributor @xiongjiwei, who made his first PR in go-service-qingstor: storage: Check if part number is valid when multipart upload.

This PR is still a draft now, and working in progress. Anyway, let's welcome @xiongjiwei !

Community#

Cooperation activities#

Summer 2021 of Open Source Promotion Plan#

This week, we got more new hands join in, and the discussion group is getting more and more active. You are welcome to keep an eye on our forum: https://forum.beyondstorage.io/, where all event-related announcements will be posted.

For more details, please refer to https://beyondstorage.io/community/events/ospp-summer-2021.