如何从puppeteer的page. evaluate()中获取流数据(node.js)

我无法重现这个问题.这个问题似乎与您正在击中的端点和/或您的服务器配置有关.我建议分享这些信息或try 建立一个你自己的再现.

这是我的复制try ，以防它对你有帮助.你可以看到代码工作，如果你运行

$ node -v
v20.11.1
$ npm i
$ node sse-endpoint &
$ node server &
$ curl localhost:3001/stream
data: {"chunk":0}

data: {"chunk":1}

data: {"chunk":2}

data: {"chunk":3}

# ... and so on, streamed every second ...

package.json:

{
  "dependencies": {
    "express": "^4.19.2",
    "puppeteer": "^22.6.0"
  }
}

sse—endpoint.js(这是你拦截的远程API的模拟):

const express = require("express");
const app = express();

app.use((req, res, next) => {
  res.setHeader("Access-Control-Allow-Origin", "*");
  next();
});

app.get("/stream", (req, res) => {
  res.writeHead(200, {
    "Connection": "keep-alive",
    "Cache-Control": "no-cache",
    "Content-Type": "text/event-stream",
  });

  let counter = 0;
  const interval = setInterval(() => {
    const chunk = JSON.stringify({chunk: counter++});
    res.write(`data: ${chunk}\n\n`);
  }, 1000);

  res.on("close", () => {
    clearInterval(interval);
    res.end();
  });
});

const listener = app.listen(process.env.PORT || 3000, () =>
  console.log(`SSE endpoint is listening on port ${listener.address().port}`)
);

server.js(这是你的API):

const express = require("express");
const puppeteer = require("puppeteer");

const app = express();
const browserReady = puppeteer.launch();

app.get("/stream", async (req, res) => {
  res.writeHead(200, {
    "Connection": "keep-alive",
    "Cache-Control": "no-cache",
    "Content-Type": "text/event-stream",
  });

  let page;
  try {
    page = await (await browserReady).newPage();
    await page.exposeFunction("writeData", data => {
      res.write(data);
    });
    await page.exposeFunction("endStream", () => {
      res.end();
    });
    await page.evaluate(async () => {
      const output = await fetch(
        "http://localhost:3000/stream"
      );
      const reader = output.body.getReader();

      while (true) {
        const {done, value} = await reader.read();

        if (done) {
          return window.endStream();
        }

        const decoder = new TextDecoder();
        const decodedData = decoder.decode(value);
        window.writeData(decodedData);
      }
    });
  } catch (err) {
    console.error(err);
    res.end();
  } finally {
    await page?.close();
  }
});

const listener = app.listen(process.env.PORT || 3001, () =>
  console.log(
    `Proxy server is listening on port ${listener.address().port}`
  )
);

注意:此代码是作为POC进行演示的，并不一定演示最佳实践.

如果SSE端点是由另一个页面流传输的，这应该不会影响这个再现.你可以让sse-endpoint服务一个HTML文件，然后在运行evaluate()/fetch()之前运行page.goto("localhost:3000")；这应该不会有什么不同.确保正确地截取或使用实际端点.很可能，问题就在那里，细节在那个阶段很重要.

取决于你想要实现的目标(我猜是像调试一个GPT聊天源？)，无论基本目标是什么，都可能有much simpler way to achieve分—这是为什么全面背景很重要的另一个原因.