Mastering AzCopy for Azure Blob Storage

AzCopy is a powerful and efficient command-line utility by Microsoft, purpose-built for transferring data to and from Azure Blob and File storage.

It is widely used by cloud engineers, architects, and administrators for bulk uploads, backup transfers, and migration tasks.

💼 Use Cases of AzCopy

- - Bulk upload of files from on-premises to Azure Blob Storage
  - Cross-region or cross-container blob transfers
  - Incremental or delta-based backups
  - Download blobs to local systems for restore or processing
  - Automated log and diagnostics uploads
  - Hybrid/offline data transfer staging
  - Headless uploads using service principal or managed identity
  - Data lake ingestion in analytics pipelines
  - Periodic sync between folders and containers

🛠️ How to Install AzCopy

AzCopy is a standalone executable — it does not require traditional installation. You can get started in minutes on both Windows and Linux systems.

✅ On Windows:

- Download the latest AzCopy release from: https://aka.ms/downloadazcopy-v10-windows
- Extract the ZIP file and locate azcopy.exe.
- Place it in a preferred folder (e.g., C:\Tools\AzCopy).
- You can either:
  - Navigate to that folder before running AzCopy, or
  - Add the folder path to your system’s PATH environment variable to run AzCopy from anywhere.
- Verify the setup:
```
azcopy --version
```

✅ On Linux:

Download from: https://aka.ms/downloadazcopy-v10-linux

Extract and install:

tar -xvf azcopy.tar.gz
sudo cp ./azcopy_linux_amd64_*/azcopy /usr/bin/

Verify:
```
azcopy --version
```

Tip : AzCopy works as a portable binary. You can run it from any folder — no installation needed, even on locked-down systems.

🔐 Understanding Authentication in AzCopy

AzCopy requires explicit authentication to access Azure Storage accounts. It operates independently from PowerShell or Azure CLI — meaning a login session from Connect-AzAccount does not carry over to AzCopy.

Important: AzCopy is a standalone tool. You must authenticate it separately, even if you are already logged in through Azure PowerShell or Azure CLI.

✅ Option 1 : Access Key (Shared Key)

$env:AZCOPY_ACCOUNT_KEY = "<your-storage-account-key>"

This sets an environment variable containing your storage account’s shared key. AzCopy will automatically use this variable for authentication in all operations that target that storage account.

Note: This is quick and script-friendly, but less secure — anyone with this key has full access to the account.

✅ Option 2 : SAS Token

You can append a SAS token to your Blob service URL:

https://<storageaccount>.blob.core.windows.net/<container>?sv=...

SAS tokens offer scoped, time-limited access. This is useful when granting temporary or limited permissions without exposing account-level keys.

✅ Option 3 : Azure AD (RBAC) – Recommended

This is the most secure and enterprise-friendly method. You authenticate AzCopy using Azure AD, and AzCopy then uses that identity to perform operations based on your RBAC role assignments on the storage account or container.

🔐 Login using Azure AD:

azcopy login

🔐 Or use a Managed Identity:

azcopy login --identity

🔐 Or use a Service Principal:

azcopy login --service-principal --application-id <appId> --tenant-id <tenantId> --client-secret <secret>

🛡️ RBAC Role Required:

To perform uploads/downloads, the identity must have one of the following roles on the storage account or container:

- Storage Blob Data Contributor (read/write)
- Storage Blob Data Reader (read-only)
- Storage Blob Data Owner (full control)

Why RBAC is Preferred :
Azure AD with RBAC provides least-privilege, auditable access. It avoids static credentials like access keys or SAS tokens and integrates well with enterprise identity controls.

🧪 Step 1 : Perform a Dry Run

Before executing a large copy operation, it’s wise to perform a dry run to validate your source and destination paths.

AzCopy will list what it would copy, but it doesn’t check the destination or compare files.

azcopy copy "\\server\share\folder\*" "https://<storageaccount>.blob.core.windows.net/<container>" --recursive=true --dry-run

This will show every file as “copy” — even if it already exists in Azure — because dry-run mode does not evaluate the destination.

📤 Step 2 : Actual Copy Command

After verifying with dry-run, remove the --dry-run flag to begin the real data transfer:

azcopy copy "\\server\share\folder\*" "https://<storageaccount>.blob.core.windows.net/<container>" --recursive=true

AzCopy will upload all files and folders it can access. Files that already exist in the destination will be skipped if they’re identical.

Note: Understanding folder\* vs folder in AzCopy

folder\* → Copies only the contents of the folder (all files and subfolders) directly into the container.
folder → Copies the entire folder (including its name) into the container as a top-level directory.

Use folder\* when you want to avoid nesting the folder itself in Azure Blob.

♻️ Step 3 : Delta Copy with Overwrite Mode

To optimize re-runs or resume interrupted transfers, use the --overwrite=ifSourceNewer flag. This uploads only if the source file is newer than what’s already in Azure.

azcopy copy "\\server\share\folder\*" "https://<storageaccount>.blob.core.windows.net/<container>" --recursive=true --overwrite=ifSourceNewer

This is ideal for incremental backups or retrying failed transfers without duplicating work.

🪵 Step 4: Check Logs

AzCopy logs all operations to a default log location: %USERPROFILE%\.azcopy\ (Windows) or ~/.azcopy/ (Linux).

For detailed logs during transfer:

azcopy copy ... --log-level=INFO --output-level=INFO

❗ Common Troubleshooting

- Access Denied at source: Ensure the Windows user running AzCopy has read permissions on the source files/folders.
- Dry Run shows all files as ‘copy’: This is expected — dry-run does not compare destination content.
- Mapped drives fail: Use UNC paths like \\server\share instead of mapped drives (e.g., Z:\).
- Authentication issues: Always check if AzCopy has been logged in separately. Use azcopy login --help to verify method.
- Incomplete transfers: Use azcopy jobs list and azcopy jobs resume <job-id> to recover.

📊 Summary Table

Task	Command	Notes
Login with access key	`$env:AZCOPY_ACCOUNT_KEY = "..."`	Set once per session
Dry run	`azcopy copy ... --dry-run`	Simulates copy only
Full copy	`azcopy copy ... --recursive=true`	Performs actual upload
Delta copy	`--overwrite=ifSourceNewer`	Only upload newer files
View logs	`%USERPROFILE%\.azcopy\`	Includes errors, skipped files
Resume job	`azcopy jobs resume <job-id>`	Resume interrupted transfer

Things AzCopy Cannot Do

❌ Copy NTFS permissions or Windows ACLs

❌ Sync bidirectionally (AzCopy is one-way only: source → destination)

❌ Copy data between Azure Storage and non-Azure cloud providers (e.g., AWS S3, Google Cloud)

❌ Bypass NTFS/share-level permissions on Windows file systems

❌ Schedule transfers on its own (no built-in scheduling or automation)

❌ Copy from Azure Files to Azure Blob or vice versa — you must download then re-upload

❌ Handle complex filtering or file transformations during transfer

Scenarios Where AzCopy Works, But Better Alternatives Exist

⚠️ Long-term or recurring sync between folders
→ Consider using Azure Data Factory or Azure File Sync for automated pipelines and stateful tracking.

⚠️ Scheduled or event-triggered transfers
→ Use PowerShell + Task Scheduler, Azure Automation, or Logic Apps for better control and scheduling.

⚠️ Moving millions of small files
→ AzCopy can be slower with high file count. Tools like Azure Data Box or Storage Migration Service may offer better performance.

⚠️ Large-scale migrations with transformations or metadata mapping
→ Use Azure Data Factory or third-party ETL tools to manage transformation logic.

⚠️ Granular access control and delegated operations
→ AzCopy is a CLI tool. For delegated workflows or portal-based access, use Azure Storage Explorer or role-based APIs.

✅ Final Thoughts

AzCopy is a highly reliable and performant tool for uploading and downloading large volumes of data to Azure Blob Storage.

The key to success lies in understanding its authentication mechanism, using dry-run correctly, and leveraging overwrite options for efficiency.

Always monitor logs, validate access permissions, and isolate issues early with incremental copy strategies.