本操作步骤中，以下配置为最终目标。
- FireCrowl (API 服务器 & 工作进程)安装到 Ubuntu 24 上
- Node.js 在整个系统范围内安装，并全局使用
pnpm命令 - 引入 Rust 工具链（rustup 等），并构建 FireCrawl 的 Rust 编写的 HTML 转换库（html-transformer）
- 安装 Playwright 的依赖包，并编写自定义的 Playwright 微服务脚本
- 创建用于启动 FireCrowl 的 systemd 服务 共 2 个（服务器用 & 工作进程用），
- 系统启动时自动启动
- 可手动执行
sudo systemctl restart firecrowl-server/sudo systemctl restart firecrowl-workers等命令
(Optional) +-----------------+ | Dify | +-----------------+ | | HTTP / REST API v +------------------------------------------------------------------+ | FireCrawl (Node.js, TypeScript, pnpm) | | Directory: /home/firecrawl/apps/api | | | | +----------------------+ (pnpm run start) | | | API Server |-----------------------------------------+ | | - Express | | | | - BullMQ Dashboard | | | +----------------------+ | | ^ | | | (Queue tasks) | | +----------------------+ (pnpm run workers) | | | Workers |-----------------------------------------+ | | - Scraping, parsing, | | | | indexing | | | +----------------------+ | | | | - .env 中设置的各项选项（API 密钥、PORT、HOST、 | | PLAYWRIGHT_MICROSERVICE_URL等） | | | | - Rust HTML Transformer: | | -> Built via Cargo in: | | /home/firecrawl/apps/api/sharedLibs/html-transformer | | -> Generates: libhtml_transformer.so | | -> Used by FireCrawl for fast HTML parsing (or falls back to | | Cheerio) | +------------------------------------------------------------------+ | | Task Queue / Rate Limit v +----------------------------------+ | Redis (localhost) | | - Used by BullMQ for job queuing| +----------------------------------+ +---------------------------------------------+ | systemd (Ubuntu 24) | | | | +--------------------------+ | | | firecrowl-server | | | | - ExecStart=pnpm run start| | | +--------------------------+ | | +--------------------------+ | | | firecrowl-workers | | | | - ExecStart=pnpm run | | | | workers | | | +--------------------------+ | | +--------------------------+ | | | firecrowl-playwright | | | | - ExecStart=pnpm run | | | | playwright-service | | | +--------------------------+ | | (Automatic start at boot, process monitoring,| | logs managed via systemd) | +---------------------------------------------+前提条件：
- Ubuntu 24（代号 “noble”）上，不使用 Docker 运行
- OS 用户具备管理员权限，可进行操作
- 假设将 FireCrowl 仓库克隆到
/home/firecrawl目录下 - Redis 已安装（
sudo apt install -y redis-server）并处于运行中
STEP1. 系统准备与 Node.js / pnpm 的安装 #
STEP1.1 系统更新与开发工具安装 #
sudo apt update sudo apt install -y build-essential pkg-config curl git libssl-devSTEP1.2 通过 NodeSource 安装 Node.js #
curl -fsSL https://deb.nodesource.com/setup_20.x | sudo -E bash - sudo apt-get install -y nodejsSTEP1.3 全局安装 pnpm #
sudo npm install -g pnpm※
which pnpm命令确认执行路径（例如：/usr/local/bin/pnpm）。
STEP2. 克隆 FireCrawl 仓库并安装依赖包 #
STEP2.1 克隆 FireCrawl #
cd /home/firecrawl git clone https://github.com/mendableai/firecrawl.gitSTEP2.2 安装依赖包 #
cd /home/firecrawl/apps/api pnpm installSTEP2.3 配置 .env 文件 #
/home/firecrawl/apps/api/.env创建或编辑该文件，设置必要的环境变量。示例：# ===== Required ENVS ====== NUM_WORKERS_PER_QUEUE=8 PORT=3002 HOST=0.0.0.0 REDIS_URL=redis://localhost:6379 REDIS_RATE_LIMIT_URL=redis://localhost:6379 PLAYWRIGHT_MICROSERVICE_URL=http://localhost:3000/html USE_DB_AUTHENTICATION=false # ===== Optional ENVS ====== TEST_API_KEY=fc-bestnet BULL_AUTH_KEY=fc-bestnet ...（其他选项按需添加）
STEP3. 引入 Rust 工具链并构建 HTML Transformer #
STEP3.1 安装 Rust 工具链 #
curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh source $HOME/.cargo/env rustc --version cargo --versionSTEP3.2 确认 FireCrawl 内的 Rust 库目录 #
在仓库根目录执行以下命令，
Cargo.toml查找存在该文件的目录。cd /home/firecrawl find . -type f -name Cargo.toml | grep -i html-transformer例如：
./sharedLibs/html-transformer/Cargo.toml若找到该文件，则进入该目录。STEP3.3 构建 Rust 库 #
cd /home/firecrawl/sharedLibs/html-transformer cargo build --release构建成功后会生成
target/release/libhtml_transformer.so。确认：ls target/release/libhtml_transformer.soSTEP3.4 配置库文件或设置环境变量 #
- 方法 A： 若 FireCrawl 的 Node.js 代码以相对路径读取，则无需额外操作
- 方法 B： 根据需要添加到
LD_LIBRARY_PATH中
export LD_LIBRARY_PATH=/home/firecrawl/sharedLibs/html-transformer/target/release:$LD_LIBRARY_PATH若要通过 systemd 自动设置，请在服务文件中
Environment=LD_LIBRARY_PATH=...添加此项。
STEP4. 设置 Playwright 并编写微服务脚本 #
STEP4.1 安装 Playwright 依赖包 #
Ubuntu 24.04 所需依赖库示例：
sudo apt-get install -y libnss3 libnspr4 libatk1.0-0 libatk-bridge2.0-0 libcups2 \ libdrm2 libxkbcommon0 libgtk-3-0 libpango-1.0-0 libcairo2 libgdk-pixbuf2.0-0 \ libgbm1 libatspi2.0-0 libx11-xcb1 libxcomposite1 libxdamage1 libxfixes3 \ libxrandr2 libxrender1 libxtst6 libxcb1 libxi6 libxcursor1 ca-certificates \ fonts-liberation xdg-utilsSTEP4.2 安装 Playwright 浏览器 #
pnpm exec playwright installSTEP4.3 编写 Playwright 微服务脚本 #
/home/firecrawl/apps/api中创建playwright-service.js，并粘贴以下代码示例。// playwright-service.js const http = require('http'); const { chromium } = require('playwright'); const PORT = 3000; const server = http.createServer(async (req, res) => { if (req.method === 'GET' && req.url.startsWith('/html')) { try { const urlParam = new URL(req.url, `http://localhost:${PORT}`).searchParams.get('url'); if (!urlParam) { res.writeHead(400, { 'Content-Type': 'text/plain' }); return res.end('Missing ?url parameter'); } const browser = await chromium.launch({ headless: true }); const page = await browser.newPage(); await page.goto(urlParam, { waitUntil: 'networkidle' }); const content = await page.content(); res.writeHead(200, { 'Content-Type': 'text/html; charset=utf-8' }); res.end(content); await browser.close(); } catch (err) { console.error('Playwright microservice error:', err); res.writeHead(500, { 'Content-Type': 'text/plain' }); res.end('Playwright error occurred'); } } else { res.writeHead(404, { 'Content-Type': 'text/plain' }); res.end('Not found'); } }); server.listen(PORT, '0.0.0.0', () => { console.log(`Playwright microservice listening on http://0.0.0.0:${PORT}/html`); });STEP4.4 在 package.json 中添加脚本 #
/home/firecrawl/apps/api/package.json的"scripts"中添加如下内容（与现有项合并）。"playwright-service": "node playwright-service.js"
STEP5. 配置 Systemd 服务 #
为使 FireCrawl 服务器、工作进程、Playwright 微服务能够自动启动，创建 systemd 单元文件。
STEP5.1 FireCrawl 服务器服务 (/etc/systemd/system/firecrowl-server.service) #
[Unit] Description=FireCrowl Server After=network.target [Service] User=firecrawl Group=firecrawl WorkingDirectory=/home/firecrawl/apps/api Environment=PATH=/usr/local/bin:/usr/bin:/bin ExecStart=/bin/bash -c 'pnpm run start' Restart=always RestartSec=5 Type=simple [Install] WantedBy=multi-user.targetSTEP5.2 FireCrawl 工作进程服务 (/etc/systemd/system/firecrowl-workers.service) #
[Unit] Description=FireCrowl Workers After=network.target [Service] User=firecrawl Group=firecrawl WorkingDirectory=/home/firecrawl/apps/api Environment=PATH=/usr/local/bin:/usr/bin:/bin ExecStart=/bin/bash -c 'pnpm run workers' Restart=always RestartSec=5 Type=simple [Install] WantedBy=multi-user.targetSTEP5.3 Playwright 微服务服务 (/etc/systemd/system/firecrowl-playwright.service) #
[Unit] Description=FireCrowl Playwright Microservice After=network.target [Service] User=firecrawl Group=firecrawl WorkingDirectory=/home/firecrawl/apps/api Environment=PATH=/home/firecrawl/.nvm/versions/node/v20.18.3/bin:/usr/local/bin:/usr/bin:/bin ExecStart=/bin/bash -c 'pnpm run playwright-service' Restart=always RestartSec=5 Type=simple [Install] WantedBy=multi-user.target注意：
User=firecrawl时，请预先创建firecrawl用户，并通过sudo chown -R firecrawl:firecrawl /home/firecrawl等命令设置好所有权。Environment=PATH=...中配置的是通过 nvm 安装的 Node.js 路径，请根据实际环境进行修改。- 若未以相对路径读取 Rust 库，则还需要设置
LD_LIBRARY_PATH。
STEP5.4 启用并启动 Systemd 服务 #
sudo systemctl daemon-reload sudo systemctl enable firecrowl-server sudo systemctl enable firecrowl-workers sudo systemctl enable firecrowl-playwright sudo systemctl start firecrowl-server sudo systemctl start firecrowl-workers sudo systemctl start firecrowl-playwrightsudo systemctl status <service>确认各服务是否处于active (running)状态。
STEP6. 重启 FireCrawl 并验证 #
- .env 配置确认：
请特别确认PLAYWRIGHT_MICROSERVICE_URL是否设置为http://localhost:3000/html。 - FireCrawl 服务器与工作进程已启动：
若日志中出现“Scrape via fetch…”等内容，则说明 HTTP 请求运行正常 - Playwright 服务的运行确认：
在浏览器中访问http://<服务器IP>:3000/html?url=https://example.com，如果返回 HTML 内容即为正常
STEP7. 最终确认 #
- Rust 库：
target/release/libhtml_transformer.so存在，且日志中没有报错 - Playwright 微服务：
测试是否正常启动并完成 JavaScript 渲染 - 整体联动：
FireCrawl 服务器、工作进程、Playwright 是否协同工作，能否对来自 Dify 等的请求正常响应，作最终检查
