Can you really throttle Azure Blob Copying?

E

EngrRob

Guest
Just saw GS RichCopy has the ability to throttle jobs (cloud) so it does not consume our bandwidth. Is that true?

The reason I am asking is we are in the process of using a different software to copy data to Azure Blob and our network bandwidth is consumed by as much as 90% by that one single server. It is impacting the rest of the servers in the datacenter.

The quesiton is, can you set it to throttle so that it does not consume more than what we give it? If so, we are looking to purchase it tonight. Please let us know ASAP, we are in the middle of the migration.
 
Hello Rob,

Thank you for reaching out. Yes, you are able to throttle cloud jobs so that they cannot consume more than what you allocate them to.

You can simply set the throttle to how much you want to cap it at and it will be restricted to it. It is recommended that each thread you use in the copy to have 4MBps. So if you have a job with 5 threads, then you should allow 20MBps (that is 160mbps).

This setting would apply to all cloud jobs running. So if you have multiple jobs copying to the cloud, it would throttle all of them at once (jobs running as a service have their own throttle as well).

The reason we had to implement this setting is that Azure Blob APIs and AWS S3 APIs are powerful and they do have the tendency to consume as much bandwidth as possible. The APIs themselves do not have a throttling mechanism so we had to implement this feature.

Please let us know if this answers your question\concern.

See screenshot below
1611182498036.png
 
That is exactly what we are looking for. We have been struggling with this for weeks and we could not find a proper workable solution. We opened a ticket with Microsoft about how we can throttle the Azure Blob API and they said it was not possible.

I have just purchased a license to test. If this works, we will be getting to portable 10 pack to use it for the rest of the servers.
 
That is correct. When we built Azure Blob integration, many of our pilot customers reported the overuse of bandwidth. So, we had to build some mechanism to control it (throttling).

Please keep us updated on how it works out for you and if you have any questions.

Thank you,
Ahmed
 
It is working exactly as advertised. :) We just gave it a test run and it is working very well. Never went over what we told it to and the network guys confirmed that from their end as well.

Seriously, this is a solid product. We were scrambling for a solution.
 
This is exactly what we had hoped to hear. Let us know if you need anything else.

Thank you,
Ahmed
 
You must log in or register to reply here.
Back
Top