feat: Add multi-threaded concurrent download support
- Add ThreadPoolExecutor for parallel attachment downloads - Add --max-workers parameter to control concurrency (default: 5) - Implement thread-safe logging with Lock mechanism - Refactor _do_download to use concurrent.futures - Add _download_single_file and _download_single_link helper functions - Update CLAUDE.md with multi-threading documentation Performance improvements: - File attachments (OData) now download in parallel - Link attachments (Scrapling) now download in parallel - Configurable worker threads for different network conditions
This commit is contained in:
15
CLAUDE.md
15
CLAUDE.md
@@ -19,7 +19,7 @@ This is a SAP C4C (Cloud for Customer) attachment downloader toolkit that retrie
|
||||
2. Fetches ServiceRequest attachments via OData endpoints:
|
||||
- `/sap/c4c/odata/v1/c4codata` - Standard C4C OData API
|
||||
- `/sap/c4c/odata/cust/v1/custticketapi` - Custom ticket API
|
||||
3. Downloads two types of attachments:
|
||||
3. Downloads two types of attachments using **multi-threaded concurrent downloads**:
|
||||
- **File attachments** (CategoryCode=2): Downloaded via OData `$value` endpoint
|
||||
- **Link attachments** (CategoryCode=3): External Salesforce links scraped using Scrapling + Playwright
|
||||
4. Handles XIssueItem-level attachments via `BO_XSRIssueItemAttachmentFolder`
|
||||
@@ -30,6 +30,11 @@ This is a SAP C4C (Cloud for Customer) attachment downloader toolkit that retrie
|
||||
- `scrapling[all]` - Web scraping framework with stealth capabilities
|
||||
- `playwright` - Browser automation for downloading Salesforce attachments
|
||||
|
||||
**Performance features:**
|
||||
- Multi-threaded concurrent downloads (default: 5 threads, configurable via `--max-workers`)
|
||||
- Thread-safe output logging with lock mechanism
|
||||
- Parallel processing of both file and link attachments
|
||||
|
||||
**Output modes:**
|
||||
- Human-readable console output (default)
|
||||
- JSON mode (`--json`) for programmatic consumption
|
||||
@@ -71,6 +76,14 @@ python sap-c4c-AttachmentFolder.py \
|
||||
--password xxx \
|
||||
--ticket 24588
|
||||
|
||||
# Download with custom thread count (default: 5)
|
||||
python sap-c4c-AttachmentFolder.py \
|
||||
--tenant https://xxx.c4c.saphybriscloud.cn \
|
||||
--user admin \
|
||||
--password xxx \
|
||||
--ticket 24588 \
|
||||
--max-workers 10
|
||||
|
||||
# Download with DSM upload
|
||||
python sap-c4c-AttachmentFolder.py \
|
||||
--tenant https://xxx.c4c.saphybriscloud.cn \
|
||||
|
||||
Reference in New Issue
Block a user