refactor: 优化代码结构并修复类型问题

- 移除未使用的TabPane组件 - 修复类型定义和导入方式 - 优化mock数据源的环境变量判断逻辑 - 更新文档结构并归档旧文件 - 添加新的UI组件和Memo组件 - 调整API路径和响应处理
2026-03-23 12:41:35 +08:00
parent a037843851
commit 2b86715c09
363 changed files with 39305 additions and 40622 deletions
--- a/docs/ARCHIVE/04_Plugin/00_Plugin_Index.md
+++ b/docs/ARCHIVE/04_Plugin/00_Plugin_Index.md
@@ -0,0 +1,57 @@
+# Node Agent 文档索引
+
+> **模块**: 04_Plugin → Node Agent (Playwright 自动化)  
+> **更新日期**: 2026-03-20
+
+---
+
+## ⚠️ 架构变更说明
+
+**Extension (浏览器插件) 已废弃**，由 **Node Agent (Playwright 自动化)** 替代。
+
+| 对比项 | Extension (旧) | Node Agent (新) |
+|--------|---------------|-----------------|
+| 运行环境 | 浏览器内 | 独立进程 |
+| 自动化引擎 | Chrome Extension API | Playwright |
+| 反检测能力 | 受限 | 完整指纹控制 |
+| 并发能力 | 单标签 | 多浏览器实例 |
+| 任务调度 | 简单消息 | Hub 拉取模式 |
+
+---
+
+## 核心文档
+
+| 文档 | 描述 | 状态 |
+|------|------|------|
+| [01_NodeAgent_Design](01_NodeAgent_Design.md) | Node Agent 架构设计 | ✅ |
+| [02_DOM_Interaction](02_DOM_Interaction.md) | DOM交互规范 | ✅ |
+
+---
+
+## 架构图
+
+```
+┌─────────────┐      ┌─────────────┐      ┌─────────────┐
+│   Server    │◄────►│    Hub      │◄────►│ Node-Agent  │
+│  (主控端)    │      │ (任务调度)   │      │ (Playwright)│
+└─────────────┘      └─────────────┘      └─────────────┘
+                                                  │
+                                           ┌──────▼──────┐
+                                           │  Chromium   │
+                                           │  (无API平台) │
+                                           └─────────────┘
+```
+
+---
+
+## 关联模块
+
+- [架构模块](../01_Architecture/00_Architecture_Index.md)
+- [后端模块](../02_Backend/00_Backend_Index.md)
+
+---
+
+## 最近更新
+
+- 2026-03-20: Extension 废弃，迁移至 Node Agent
+- 2026-03-19: 重构插件文档结构，统一命名规范
--- a/docs/ARCHIVE/04_Plugin/01_NodeAgent_Design.md
+++ b/docs/ARCHIVE/04_Plugin/01_NodeAgent_Design.md
@@ -0,0 +1,263 @@
+# Node Agent 设计文档
+
+> **定位**: Win Node Agent - 无 API 平台执行代理  
+> **更新日期**: 2026-03-20  
+> **最高优先级参考**: [Business_ClosedLoops.md](../00_Business/Business_ClosedLoops.md)
+
+---
+
+## 1. 概述
+
+### 1.1 背景
+
+Extension (浏览器插件) 方案存在以下局限性：
+- Manifest V3 权限限制
+- 反检测能力不足
+- 无法多实例并发
+- 任务调度不可靠
+
+**Node Agent** 基于 Playwright 构建，提供更强大的自动化能力。
+
+### 1.2 核心能力
+
+| 能力 | 说明 |
+|------|------|
+| **无 API 平台采集** | TikTok Shop, Temu, 1688 等 |
+| **自动化操作** | 刊登、订单处理、广告投放 |
+| **反检测** | 指纹隔离、代理 IP、浏览器上下文隔离 |
+| **多实例并发** | 支持多店铺同时运行 |
+
+---
+
+## 2. 技术栈
+
+| 层级 | 技术 | 版本 | 用途 |
+|------|------|------|------|
+| **Runtime** | Node.js | 18+ | 运行环境 |
+| **Automation** | Playwright | 1.58+ | 浏览器自动化 |
+| **Language** | TypeScript | 5.x | 开发语言 |
+| **Communication** | Axios | - | HTTP 通信 |
+
+---
+
+## 3. 架构设计
+
+### 3.1 系统架构
+
+```
+┌─────────────────────────────────────────────────────────────┐
+│                        Crawlful Hub                         │
+├─────────────────────────────────────────────────────────────┤
+│                                                              │
+│  ┌──────────────┐     ┌──────────────┐     ┌──────────────┐ │
+│  │   Server     │     │     Hub      │     │  Dashboard   │ │
+│  │  (业务逻辑)   │◄───►│  (任务调度)   │◄───►│   (控制台)    │ │
+│  └──────────────┘     └──────┬───────┘     └──────────────┘ │
+│                              │                              │
+│                              ▼                              │
+│                    ┌──────────────────┐                     │
+│                    │   Node Agent     │                     │
+│                    │   (Playwright)   │                     │
+│                    └────────┬─────────┘                     │
+│                             │                               │
+│         ┌───────────────────┼───────────────────┐          │
+│         ▼                   ▼                   ▼          │
+│  ┌──────────────┐   ┌──────────────┐   ┌──────────────┐   │
+│  │  Browser #1  │   │  Browser #2  │   │  Browser #N  │   │
+│  │  (店铺A)      │   │  (店铺B)      │   │  (店铺N)      │   │
+│  └──────────────┘   └──────────────┘   └──────────────┘   │
+│                                                              │
+└─────────────────────────────────────────────────────────────┘
+```
+
+### 3.2 核心类设计
+
+```typescript
+export class NodeAgent {
+  private hubUrl: string;
+  private nodeId: string;
+  private heartbeatInterval: number = 30000;
+  private pollInterval: number = 10000;
+
+  async start(): Promise<void>;
+  async register(): Promise<void>;
+  async heartbeat(): Promise<void>;
+  async pollTasks(): Promise<void>;
+  async executeTask(task: NodeTask): Promise<void>;
+  async reportReceipt(receipt: any): Promise<void>;
+}
+```
+
+### 3.3 任务类型
+
+```typescript
+export enum TaskType {
+  COLLECT_PRODUCT = 'COLLECT_PRODUCT',
+  COLLECT_ORDER = 'COLLECT_ORDER',
+  PUBLISH_PRODUCT = 'PUBLISH_PRODUCT',
+  PROCESS_ORDER = 'PROCESS_ORDER',
+  SYNC_INVENTORY = 'SYNC_INVENTORY',
+  MANAGE_AD = 'MANAGE_AD',
+}
+```
+
+---
+
+## 4. 核心功能
+
+### 4.1 节点注册
+
+```typescript
+private async register() {
+  await axios.post(`${this.hubUrl}/api/v1/nodes/register`, {
+    nodeId: this.nodeId,
+    version: '1.0.0',
+    os: os.type(),
+    hostname: os.hostname(),
+    shops: []
+  });
+}
+```
+
+### 4.2 任务轮询
+
+```typescript
+private async pollTasks() {
+  const response = await axios.get(
+    `${this.hubUrl}/api/v1/nodes/pull-task?nodeId=${this.nodeId}`
+  );
+  
+  if (response.data?.task) {
+    await this.executeTask(response.data.task);
+  }
+}
+```
+
+### 4.3 任务执行
+
+```typescript
+private async executeTask(task: NodeTask) {
+  try {
+    const browser = await chromium.launch({
+      headless: false,
+      proxy: { server: task.proxyUrl }
+    });
+    
+    const context = await browser.newContext({
+      viewport: { width: 1920, height: 1080 }
+    });
+    
+    const page = await context.newPage();
+    
+    // 执行具体任务...
+    
+    await browser.close();
+    
+    await this.reportReceipt({
+      taskId: task.id,
+      traceId: task.traceId,
+      status: 'success',
+      resultData: {}
+    });
+  } catch (err) {
+    await this.reportReceipt({
+      taskId: task.id,
+      traceId: task.traceId,
+      status: 'failed',
+      errorMessage: err.message
+    });
+  }
+}
+```
+
+---
+
+## 5. 店铺隔离策略
+
+### 5.1 一店一上下文
+
+```typescript
+interface ShopContext {
+  shopId: string;
+  profileDir: string;
+  proxy: ProxyConfig;
+  fingerprintPolicy: FingerprintConfig;
+}
+```
+
+### 5.2 同店任务串行
+
+```typescript
+class TaskQueue {
+  private queues: Map<string, TaskQueue> = new Map();
+  
+  async addTask(shopId: string, task: Task) {
+    if (!this.queues.has(shopId)) {
+      this.queues.set(shopId, new TaskQueue());
+    }
+    await this.queues.get(shopId).add(task);
+  }
+}
+```
+
+---
+
+## 6. 反检测策略
+
+| 策略 | 说明 |
+|------|------|
+| **指纹隔离** | 每个店铺独立浏览器指纹 |
+| **代理 IP** | 每个店铺独立代理 |
+| **行为模拟** | 随机延迟、鼠标轨迹 |
+| **User-Agent** | 随机 UA 轮换 |
+
+---
+
+## 7. 目录结构
+
+```
+node-agent/
+├── src/
+│   ├── main.ts              # 入口
+│   ├── index.ts             # NodeAgent 类
+│   ├── tasks/               # 任务处理器
+│   │   ├── collectProduct.ts
+│   │   ├── publishProduct.ts
+│   │   └── processOrder.ts
+│   ├── platforms/           # 平台适配器
+│   │   ├── tiktok.ts
+│   │   ├── temu.ts
+│   │   └── ali1688.ts
+│   └── utils/               # 工具函数
+│       ├── logger.ts
+│       └── fingerprint.ts
+├── package.json
+└── tsconfig.json
+```
+
+---
+
+## 8. 配置
+
+```typescript
+interface AgentConfig {
+  hubUrl: string;
+  nodeId: string;
+  heartbeatInterval: number;
+  pollInterval: number;
+  maxConcurrentBrowsers: number;
+  proxyPool: ProxyConfig[];
+}
+```
+
+---
+
+## 9. 相关文档
+
+- [DOM Interaction](02_DOM_Interaction.md)
+- [Backend Design](../02_Backend/Backend_Design.md)
+- [Business ClosedLoops](../00_Business/Business_ClosedLoops.md)
+
+---
+
+*本文档基于业务闭环设计，最后更新: 2026-03-20*
--- a/docs/ARCHIVE/04_Plugin/02_DOM_Interaction.md
+++ b/docs/ARCHIVE/04_Plugin/02_DOM_Interaction.md
@@ -0,0 +1,475 @@
+# DOM Interaction (Crawlful Hub Plugin)
+
+> **定位**：Crawlful Hub 插件 DOM 交互文档 - 描述如何与电商平台页面进行 DOM 交互。
+> **更新日期**: 2026-03-18
+
+---
+
+## 1. DOM 选择器策略
+
+### 1.1 选择器类型
+
+| 类型 | 示例 | 适用场景 |
+|------|------|----------|
+| ID | `#productTitle` | 唯一元素 |
+| Class | `.a-price-whole` | 样式类 |
+| Attribute | `[data-asin]` | 数据属性 |
+| XPath | `//span[@id='price']` | 复杂结构 |
+| CSS Selector | `div.product > h1` | 层级关系 |
+
+### 1.2 平台选择器映射
+
+#### Amazon
+
+```typescript
+const amazonSelectors = {
+  product: {
+    title: '#productTitle',
+    price: '.a-price-whole, .a-price .a-offscreen',
+    listPrice: '.a-text-price .a-offscreen',
+    images: '#landingImage, #imgTagWrapperId img',
+    description: '#feature-bullets ul, #productDescription',
+    brand: '#bylineInfo',
+    asin: '[data-asin]',
+    rating: '#acrPopover .a-icon-alt',
+    reviewCount: '#acrCustomerReviewText',
+    availability: '#availability span',
+    category: '#wayfinding-breadcrumbs_container ul',
+  },
+  search: {
+    results: '[data-component-type="s-search-result"]',
+    title: 'h2 a span',
+    price: '.a-price-whole',
+    image: '.s-image',
+    rating: '.a-icon-alt',
+  },
+  seller: {
+    name: '#merchant-info a:first-child',
+    rating: '#merchant-info .a-icon-alt',
+  },
+};
+```
+
+#### eBay
+
+```typescript
+const ebaySelectors = {
+  product: {
+    title: 'h1[data-testid="x-item-title-label"]',
+    price: '.notranslate.vi-price .notranslate',
+    images: '#icImg, .vi-image-gallery__image',
+    description: '#desc_wrapper, #ds_div',
+    condition: '.u-flL.condText',
+    seller: '.mbg-nw',
+    sellerRating: '.mbg-l .mbg-fb',
+    quantity: '#qtyTextBox',
+    shipping: '#fshippingCost span',
+  },
+  search: {
+    results: '.s-item',
+    title: '.s-item__title',
+    price: '.s-item__price',
+    image: '.s-item__image img',
+  },
+};
+```
+
+#### Shopify
+
+```typescript
+const shopifySelectors = {
+  product: {
+    title: 'h1.product-title, h1[data-product-title]',
+    price: '.product-price, [data-product-price]',
+    comparePrice: '.compare-price, [data-compare-price]',
+    images: '.product-image, .product__media img',
+    description: '.product-description, [data-product-description]',
+    variants: '[data-variant-id]',
+    inventory: '[data-inventory]',
+    sku: '[data-sku]',
+    barcode: '[data-barcode]',
+  },
+  admin: {
+    products: 'table tbody tr',
+    orders: 'table tbody tr',
+    title: 'td:first-child a',
+    status: 'td:nth-child(3) span',
+  },
+};
+```
+
+---
+
+## 2. DOM 操作工具
+
+### 2.1 元素提取工具
+
+```typescript
+// src/content/utils/domUtils.ts
+
+export class DOMUtils {
+  /**
+   * 安全地获取元素文本
+   */
+  static getText(selector: string, context: Document | Element = document): string {
+    const element = context.querySelector(selector);
+    return element?.textContent?.trim() || '';
+  }
+
+  /**
+   * 获取元素属性
+   */
+  static getAttr(selector: string, attr: string, context: Document | Element = document): string {
+    const element = context.querySelector(selector);
+    return element?.getAttribute(attr) || '';
+  }
+
+  /**
+   * 获取多个元素的文本列表
+   */
+  static getTextList(selector: string, context: Document | Element = document): string[] {
+    const elements = context.querySelectorAll(selector);
+    return Array.from(elements).map(el => el.textContent?.trim() || '');
+  }
+
+  /**
+   * 获取图片 URL 列表
+   */
+  static getImageUrls(selector: string, context: Document | Element = document): string[] {
+    const images = context.querySelectorAll(selector);
+    return Array.from(images)
+      .map(img => {
+        const src = img.getAttribute('src') || img.getAttribute('data-src');
+        return src ? this.resolveUrl(src) : '';
+      })
+      .filter(Boolean);
+  }
+
+  /**
+   * 等待元素出现
+   */
+  static waitForElement(
+    selector: string,
+    timeout: number = 10000,
+    context: Document | Element = document
+  ): Promise<Element | null> {
+    return new Promise((resolve) => {
+      const element = context.querySelector(selector);
+      if (element) {
+        resolve(element);
+        return;
+      }
+
+      const observer = new MutationObserver(() => {
+        const element = context.querySelector(selector);
+        if (element) {
+          observer.disconnect();
+          resolve(element);
+        }
+      });
+
+      observer.observe(document.body, {
+        childList: true,
+        subtree: true,
+      });
+
+      setTimeout(() => {
+        observer.disconnect();
+        resolve(null);
+      }, timeout);
+    });
+  }
+
+  /**
+   * 解析相对 URL
+   */
+  private static resolveUrl(url: string): string {
+    if (url.startsWith('http')) return url;
+    if (url.startsWith('//')) return `https:${url}`;
+    return new URL(url, window.location.href).href;
+  }
+}
+```
+
+### 2.2 事件监听工具
+
+```typescript
+// src/content/utils/eventUtils.ts
+
+export class EventUtils {
+  /**
+   * 监听页面变化
+   */
+  static onPageChange(callback: (url: string) => void): void {
+    let currentUrl = window.location.href;
+    
+    const observer = new MutationObserver(() => {
+      if (window.location.href !== currentUrl) {
+        currentUrl = window.location.href;
+        callback(currentUrl);
+      }
+    });
+
+    observer.observe(document.body, {
+      childList: true,
+      subtree: true,
+    });
+
+    // 同时监听 popstate 事件
+    window.addEventListener('popstate', () => {
+      callback(window.location.href);
+    });
+  }
+
+  /**
+   * 监听元素出现
+   */
+  static onElementAppear(
+    selector: string,
+    callback: (element: Element) => void
+  ): void {
+    const observer = new MutationObserver((mutations) => {
+      mutations.forEach((mutation) => {
+        mutation.addedNodes.forEach((node) => {
+          if (node instanceof Element) {
+            if (node.matches(selector)) {
+              callback(node);
+            }
+            node.querySelectorAll(selector).forEach(callback);
+          }
+        });
+      });
+    });
+
+    observer.observe(document.body, {
+      childList: true,
+      subtree: true,
+    });
+  }
+
+  /**
+   * 模拟用户点击
+   */
+  static simulateClick(element: Element): void {
+    const event = new MouseEvent('click', {
+      bubbles: true,
+      cancelable: true,
+      view: window,
+    });
+    element.dispatchEvent(event);
+  }
+
+  /**
+   * 模拟输入
+   */
+  static simulateInput(element: HTMLInputElement, value: string): void {
+    element.value = value;
+    element.dispatchEvent(new Event('input', { bubbles: true }));
+    element.dispatchEvent(new Event('change', { bubbles: true }));
+  }
+}
+```
+
+---
+
+## 3. 平台特定交互
+
+### 3.1 Amazon 交互
+
+```typescript
+// src/content/interactions/amazonInteraction.ts
+
+export class AmazonInteraction {
+  /**
+   * 获取商品详情
+   */
+  async getProductDetails(): Promise<ProductData> {
+    const title = DOMUtils.getText('#productTitle');
+    const priceText = DOMUtils.getText('.a-price-whole, .a-price .a-offscreen');
+    const price = this.parsePrice(priceText);
+    
+    const images = DOMUtils.getImageUrls('#landingImage, #imgTagWrapperId img');
+    const description = DOMUtils.getText('#feature-bullets ul');
+    const asin = DOMUtils.getAttr('[data-asin]', 'data-asin');
+    
+    return {
+      platform: 'AMAZON',
+      asin,
+      title,
+      price,
+      images,
+      description,
+      url: window.location.href,
+    };
+  }
+
+  /**
+   * 获取搜索列表
+   */
+  async getSearchResults(): Promise<ProductData[]> {
+    const results: ProductData[] = [];
+    const items = document.querySelectorAll('[data-component-type="s-search-result"]');
+    
+    items.forEach((item) => {
+      const title = DOMUtils.getText('h2 a span', item);
+      const priceText = DOMUtils.getText('.a-price-whole', item);
+      const price = this.parsePrice(priceText);
+      const asin = item.getAttribute('data-asin') || '';
+      
+      results.push({
+        platform: 'AMAZON',
+        asin,
+        title,
+        price,
+        url: `https://www.amazon.com/dp/${asin}`,
+      });
+    });
+    
+    return results;
+  }
+
+  /**
+   * 导航到刊登页面
+   */
+  async navigateToListing(): Promise<void> {
+    window.location.href = 'https://sellercentral.amazon.com/inventory/add';
+  }
+
+  /**
+   * 填写刊登表单
+   */
+  async fillListingForm(product: ProductData): Promise<void> {
+    // 等待页面加载
+    await DOMUtils.waitForElement('#title', 10000);
+    
+    // 填写标题
+    const titleInput = document.querySelector('#title') as HTMLInputElement;
+    if (titleInput) {
+      EventUtils.simulateInput(titleInput, product.title);
+    }
+    
+    // 填写价格
+    const priceInput = document.querySelector('#price') as HTMLInputElement;
+    if (priceInput) {
+      EventUtils.simulateInput(priceInput, product.price.toString());
+    }
+    
+    // 填写描述
+    const descInput = document.querySelector('#description') as HTMLTextAreaElement;
+    if (descInput) {
+      EventUtils.simulateInput(descInput, product.description);
+    }
+  }
+
+  private parsePrice(priceText: string): number {
+    const match = priceText.replace(/[^\d.]/g, '').match(/(\d+\.?\d*)/);
+    return match ? parseFloat(match[1]) : 0;
+  }
+}
+```
+
+### 3.2 eBay 交互
+
+```typescript
+// src/content/interactions/ebayInteraction.ts
+
+export class EbayInteraction {
+  /**
+   * 获取商品详情
+   */
+  async getProductDetails(): Promise<ProductData> {
+    const title = DOMUtils.getText('h1[data-testid="x-item-title-label"]');
+    const priceText = DOMUtils.getText('.notranslate.vi-price .notranslate');
+    const price = this.parsePrice(priceText);
+    
+    const images = DOMUtils.getImageUrls('#icImg, .vi-image-gallery__image');
+    const description = DOMUtils.getText('#desc_wrapper');
+    const itemId = this.extractItemId();
+    
+    return {
+      platform: 'EBAY',
+      itemId,
+      title,
+      price,
+      images,
+      description,
+      url: window.location.href,
+    };
+  }
+
+  private extractItemId(): string {
+    const match = window.location.pathname.match(/\/(\d+)$/);
+    return match ? match[1] : '';
+  }
+
+  private parsePrice(priceText: string): number {
+    const match = priceText.replace(/[^\d.]/g, '').match(/(\d+\.?\d*)/);
+    return match ? parseFloat(match[1]) : 0;
+  }
+}
+```
+
+---
+
+## 4. 异常处理
+
+### 4.1 选择器失效处理
+
+```typescript
+export class SelectorFallback {
+  /**
+   * 尝试多个选择器
+   */
+  static trySelectors(selectors: string[]): string {
+    for (const selector of selectors) {
+      const text = DOMUtils.getText(selector);
+      if (text) return text;
+    }
+    return '';
+  }
+
+  /**
+   * 使用 XPath 作为备选
+   */
+  static queryXPath(xpath: string): string {
+    const result = document.evaluate(
+      xpath,
+      document,
+      null,
+      XPathResult.STRING_TYPE,
+      null
+    );
+    return result.stringValue || '';
+  }
+}
+```
+
+### 4.2 页面变化检测
+
+```typescript
+export class PageChangeDetector {
+  private lastContent: string = '';
+  private checkInterval: number = 1000;
+
+  start(callback: () => void): void {
+    setInterval(() => {
+      const currentContent = document.body.innerHTML;
+      if (currentContent !== this.lastContent) {
+        this.lastContent = currentContent;
+        callback();
+      }
+    }, this.checkInterval);
+  }
+}
+```
+
+---
+
+## 5. 相关文档
+
+- [Plugin Design](./Plugin_Design.md)
+- [Automation Scripts](./Automation_Scripts.md)
+
+---
+
+*本文档基于实际平台页面结构，最后更新: 2026-03-18*