feat: Add multi-threaded concurrent download support

- Add ThreadPoolExecutor for parallel attachment downloads
- Add --max-workers parameter to control concurrency (default: 5)
- Implement thread-safe logging with Lock mechanism
- Refactor _do_download to use concurrent.futures
- Add _download_single_file and _download_single_link helper functions
- Update CLAUDE.md with multi-threading documentation

Performance improvements:
- File attachments (OData) now download in parallel
- Link attachments (Scrapling) now download in parallel
- Configurable worker threads for different network conditions
This commit is contained in:
afei A
2026-03-12 13:01:13 +08:00
parent 929d3c2ec9
commit 84273a765e
2 changed files with 103 additions and 39 deletions

View File

@@ -19,7 +19,7 @@ This is a SAP C4C (Cloud for Customer) attachment downloader toolkit that retrie
2. Fetches ServiceRequest attachments via OData endpoints:
- `/sap/c4c/odata/v1/c4codata` - Standard C4C OData API
- `/sap/c4c/odata/cust/v1/custticketapi` - Custom ticket API
3. Downloads two types of attachments:
3. Downloads two types of attachments using **multi-threaded concurrent downloads**:
- **File attachments** (CategoryCode=2): Downloaded via OData `$value` endpoint
- **Link attachments** (CategoryCode=3): External Salesforce links scraped using Scrapling + Playwright
4. Handles XIssueItem-level attachments via `BO_XSRIssueItemAttachmentFolder`
@@ -30,6 +30,11 @@ This is a SAP C4C (Cloud for Customer) attachment downloader toolkit that retrie
- `scrapling[all]` - Web scraping framework with stealth capabilities
- `playwright` - Browser automation for downloading Salesforce attachments
**Performance features:**
- Multi-threaded concurrent downloads (default: 5 threads, configurable via `--max-workers`)
- Thread-safe output logging with lock mechanism
- Parallel processing of both file and link attachments
**Output modes:**
- Human-readable console output (default)
- JSON mode (`--json`) for programmatic consumption
@@ -71,6 +76,14 @@ python sap-c4c-AttachmentFolder.py \
--password xxx \
--ticket 24588
# Download with custom thread count (default: 5)
python sap-c4c-AttachmentFolder.py \
--tenant https://xxx.c4c.saphybriscloud.cn \
--user admin \
--password xxx \
--ticket 24588 \
--max-workers 10
# Download with DSM upload
python sap-c4c-AttachmentFolder.py \
--tenant https://xxx.c4c.saphybriscloud.cn \