For three months, this business grew at the speed of manual data entry, one product every twelve minutes. Our first deployment did not introduce complexity. It removed the constraint entirely.
The Initial Problem
The client operates a growing Shopify store supported by multiple suppliers. For three months, product onboarding was handled entirely through manual input. Three staff members extracted data from supplier websites and rebuilt listings inside Shopify, including descriptions, images, variants, and structural formatting. Output averaged 40 to 50 products per day.
At the lower bound, that meant 40 products across an eight hour day. Five per hour. One every twelve minutes. With more than 20 suppliers waiting to be onboarded, catalogue expansion was capped by human throughput. The business was not constrained by demand or supplier availability. It was constrained by process.
The Tools and Architecture
We focused on automation inside the existing workflow rather than introducing new systems. V1 targeted suppliers already operating on Shopify — which represented over 50% of the supplier base — making it the highest-impact starting point. Direct data exports from suppliers and third-party API integrations were not available options, so extracting from source was the practical path. Further iterations will extend coverage to non-Shopify supplier environments.
The solution consisted of three core components:
1. API-Based Web Scraper
We built a custom API-based controlled scraper to extract product data directly from supplier Shopify environments. This produced structured, repeatable data output rather than relying on manual copy and paste.
2. Google Sheets as Control Layer
The extracted data flows into Google Sheets. The owner already uses Sheets operationally, so this became the validation interface. No new dashboard. No training overhead.
3. Google Apps Script for Transformation
A custom Google Sheet Script reformats the structured data into Matrixify’s required schema for Shopify imports. The script allows the operator to:
- Run a limited batch for testing
- Execute a full catalogue batch
- Push validated data directly to the Matrixify import sheet
Once pushed, Shopify processes the product import in the background via Matrixify. The transformation and export layer completes in approximately 5 to 15 minutes, even at scale. Shopify’s import phase can take several hours depending on dataset size, although this run imported 350 new products in around 50 minutes.
The scraping and conversion pipeline for the custom Matrixify setup supports an unlimited number of products. However, individual runs are capped at 5,000 products per execution or import cycle due to the Matrixify account limit.
Key Challenges
The technical build itself was straightforward. The operational considerations were more important, and this is how we were able to scale product uploads from 40 a day to thousands.
Challenge | Operational Risk | Our Response |
|---|---|---|
Data Normalisation | Supplier data structures were inconsistent. Variants, image hierarchies, and attribute naming lacked uniformity. | Implemented controlled transformation logic to standardise data before export. |
Error Handling at Scale | Minor formatting inconsistencies could invalidate thousands of products during import. | Embedded validation safeguards within Google Sheets prior to export. |
Maintaining Human Oversight | Fully automated imports without review increased commercial risk. | Enabled selective test runs and staged batch imports before full deployment. |
Supplier Environment Scope | Not all suppliers operate on the same platform, and direct data exports or API integrations were not available options. | V1 targets Shopify-hosted suppliers, covering the majority of the onboarding backlog. Non-Shopify environments are addressed in subsequent iterations. |
The objective was not maximum automation. It was controlled acceleration.
Measurable Outcome and Strategic Impact
The throughput shift is structural. Previously, the team processed approximately 40 products per person per day. The new system processes up to 5,000 products in 5 to 15 minutes before Shopify import. Even allowing for background processing time inside Shopify, the manual constraint has been removed. More importantly, the three staff members once assigned to data entry have been redirected to structured quality control, reviewing products at a rate of roughly one every one to two minutes. In many cases, they now quality control more products in an hour than they previously added in a full day, representing an approximate eightfold increase in productive output per team member.
The commercial impact extends beyond time savings. With more than 20 suppliers waiting to be onboarded, growth is no longer capped by labour capacity. Supplier expansion can be scheduled based on commercial priority rather than operational bandwidth. Human effort has shifted from replication to performance protection, directly influencing:
- Product accuracy
- Variant consistency
- Pricing reliability
- Search visibility within Shopify
The system did not alter the business model. It increased operational velocity and removed a growth ceiling.
Why This Matters
This first deployment reflects a core principle at Swarm Labs.
High impact projects do not need to be complex. They need to remove measurable constraints.
The bottleneck was manual product entry, so we automated it. The result was the ability to scale Shopify uploads from 40 a day to thousands.
And now that constraint has disappeared, the organisation’s growth capacity is allowed to expand immediately.
That is the standard we apply moving forward.
Product Count | Hours Recovered | Approx. Full-Time Weeks (40 hrs/week) |
|---|---|---|
500 | 100 hours | 2.5 weeks |
1,000 | 200 hours | 5 weeks |
2,500 | 500 hours | 12.5 weeks |
5,000 | 1,000 hours | 25 weeks |
Note: These figures represent recovered manual entry time only. They exclude supervision, correction cycles, and the operational drag created by repetitive data input.
We track this metric as part of our long term objective to recover one million hours of human effort through applied automation. For accuracy, we do not measure recovery at the point of initial data extraction. The recorded metric is triggered only when reviewed data is pushed to the Matrixify sheet for import. That event marks validated output, not theoretical capacity.