登顶GitHub趋势榜，标星1.8k：200行JS代码让画面人物瞬间消失
整理 | 夕颜
出品 | CSDN（ID:CSDNnews）
今天，一个名为 Real-Time-Person-Removal（实时人物去除）项目在GitHub上火了，登上近日GitHub Trending第一，目前已经获得1.8k star。
这个项目的神奇之处在于，只需要在网络浏览器中使用JavaScript，用200多行TensorFlow.js代码，就可以实时让视频画面中的人物对象从复杂的背景中凭空消失！
这虽然不能让你在现实生活中像哈利·波特一样隐身的梦想成真，但至少在视频、动画里可以体验一把隐身的快感！
首先奉上GitHub地址：
https://github.com/jasonmayes/Real-Time-Person-Removal这个项目能干啥？
本项目的作者@jasonmayes（Jason Mayes）是谷歌的一名资深开发者，是机器智能研究和高级开发的倡导者，作为一名TensorFlow.js专家，他拥有超过15年使用新技术开发创新Web解决方案的经验。
他在项目介绍中表示，这段代码的目的在于随着时间的推移学习视频背景的构成，让作者可以尝试从背景中移除任何人物，而所有效果都是使用TensorFlow.js在浏览器中实时实现的。
但同时作者表示，这只是一个实验，并非在所有情况下都是完美的。
消失的人
废话不多说，上代码！
可能有人会觉得在复杂的背景下实现“隐身”是很复杂的吧，而且还是实时的，但实际上实现这样的效果却只需要200多行JS代码：
1/**2 * @license3 * Copyright 2018 Google LLC. All Rights Reserved.4 * Licensed under the Apache License, Version 2.0 (the "License");5 * you may not use this file except in compliance with the License.6 * You may obtain a copy of the License at7 *8 * http://www.apache.org/licenses/LICENSE-2.09 *10 * Unless required by applicable law or agreed to in writing, software11 * distributed under the License is distributed on an "AS IS" BASIS,12 * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.13 * See the License for the specific language governing permissions and14 * limitations under the License.15 * =============================================================================16 */1718/********************************************************************19 * Real-Time-Person-Removal Created by Jason Mayes 2020.20 *21 * Get latest code on my Github:22 * https://github.com/jasonmayes/Real-Time-Person-Removal23 *24 * Got questions? Reach out to me on social:25 * Twitter: @jason_mayes26 * LinkedIn: https://www.linkedin.com/in/creativetech27 ********************************************************************/2829const video = document.getElementById(webcam);30const liveView = document.getElementById(liveView);31const demosSection = document.getElementById(demos);32const DEBUG = false;3334// An object to configure parameters to set for the bodypix model.35// See github docs for explanations.36const bodyPixProperties = {37 architecture: MobileNetV1,38 outputStride: 16,39 multiplier: 0.75,40 quantBytes: 441};4243// An object to configure parameters for detection. I have raised44// the segmentation threshold to 90% confidence to reduce the45// number of false positives.46const segmentationProperties = {47 flipHorizontal: false,48 internalResolution: high,49 segmentationThreshold: 0.950};5152// Must be even. The size of square we wish to search for body parts.53// This is the smallest area that will render/not render depending on54// if a body part is found in that square.55const SEARCH_RADIUS = 300;56const SEARCH_OFFSET = SEARCH_RADIUS / 2;575859// RESOLUTION_MIN should be smaller than SEARCH RADIUS. About 10x smaller seems to 60// work well. Effects overlap in search space to clean up body overspill for things61// that were not classified as body but infact were.62const RESOLUTION_MIN = 20;636465// Render returned segmentation data to a given canvas context.66function processSegmentation(canvas, segmentation) {67 var ctx = canvas.getContext(2d);6869 // Get data from our overlay canvas which is attempting to estimate background.70 var imageData = ctx.getImageData(0, 0, canvas.width, canvas.height);71 var data = imageData.data;7273 // Get data from the live webcam view which has all data.74 var liveData = videoRenderCanvasCtx.getImageData(0, 0, canvas.width, canvas.height);75 var dataL = liveData.data;7677 // Now loop through and see if pixels contain human parts. If not, update 78 // backgound understanding with new data.79 for (let x = RESOLUTION_MIN; x < canvas.width; x += RESOLUTION_MIN) {80 for (let y = RESOLUTION_MIN; y < canvas.height; y += RESOLUTION_MIN) {81 // Convert xy co-ords to array offset.82 let n = y * canvas.width + x;8384 let foundBodyPartNearby = false;8586 // Lets check around a given pixel if any other pixels were body like.87 let yMin = y - SEARCH_OFFSET;88 yMin = yMin < 0 ? 0: yMin;8990 let yMax = y + SEARCH_OFFSET;91 yMax = yMax > canvas.height ? canvas.height : yMax;9293 let xMin = x - SEARCH_OFFSET;94 xMin = xMin < 0 ? 0: xMin;9596 let xMax = x + SEARCH_OFFSET;97 xMax = xMax > canvas.width ? canvas.width : xMax;9899 for (let i = xMin; i < xMax; i++) {100 for (let j = yMin; j < yMax; j++) {101102 let offset = j * canvas.width + i;103 // If any of the pixels in the square we are analysing has a body104 // part, mark as contaminated.105 if (segmentation.data[offset] !== 0) {106 foundBodyPartNearby = true;107 break;108 } 109 }110 }111112 // Update patch if patch was clean. 113 if (!foundBodyPartNearby) {114 for (let i = xMin; i < xMax; i++) {115 for (let j = yMin; j < yMax; j++) {116 // Convert xy co-ords to array offset.117 let offset = j * canvas.width + i;118119120 data[offset * 4] = dataL[offset * 4]; 121 data[offset * 4 + 1] = dataL[offset * 4 + 1];122 data[offset * 4 + 2] = dataL[offset * 4 + 2];123 data[offset * 4 + 3] = 255; 124 }125 }126 } else {127 if (DEBUG) {128 for (let i = xMin; i < xMax; i++) {129 for (let j = yMin; j < yMax; j++) {130 // Convert xy co-ords to array offset.131 let offset = j * canvas.width + i;132133134 data[offset * 4] = 255; 135 data[offset * 4 + 1] = 0;136 data[offset * 4 + 2] = 0;137 data[offset * 4 + 3] = 255; 138 }139 } 140 }141 }142143144 }145 }146 ctx.putImageData(imageData, 0, 0);147}148149// Lets load the model with our parameters defined above.150// Before we can use bodypix class we must wait for it to finish151// loading. Machine Learning models can be large and take a moment to152// get everything needed to run.153var modelHasLoaded = false;154var model = undefined;155156model = bodyPix.load(bodyPixProperties).then(function (loadedModel) {157 model = loadedModel;158 modelHasLoaded = true;159 // Show demo section now model is ready to use.160 demosSection.classList.remove(invisible);161});162163/********************************************************************164// Continuously grab image from webcam stream and classify it.165********************************************************************/166167var previousSegmentationComplete = true;168169// Check if webcam access is supported.170function hasGetUserMedia() {171 return !!(navigator.mediaDevices &&172 navigator.mediaDevices.getUserMedia);173}174175// This function will repeatidly call itself when the browser is ready to process176// the next frame from webcam.177function predictWebcam() {178 if (previousSegmentationComplete) {179 // Copy the video frame from webcam to a tempory canvas in memory only (not in the DOM).180 videoRenderCanvasCtx.drawImage(video, 0, 0);181 previousSegmentationComplete = false;182 // Now classify the canvas image we have available.183 model.segmentPerson(videoRenderCanvas, segmentationProperties).then(function(segmentation) {184 processSegmentation(webcamCanvas, segmentation);185 previousSegmentationComplete = true;186 });187 }188189 // Call this function again to keep predicting when the browser is ready.190 window.requestAnimationFrame(predictWebcam);191}192193// Enable the live webcam view and start classification.194function enableCam(event) {195 if (!modelHasLoaded) {196 return;197 }198199 // Hide the button.200 event.target.classList.add(removed); 201202 // getUsermedia parameters.203 const constraints = {204 video: true205 };206207 // Activate the webcam stream.208 navigator.mediaDevices.getUserMedia(constraints).then(function(stream) {209 video.addEventListener(loadedmetadata, function() {210 // Update widths and heights once video is successfully played otherwise211 // it will have width and height of zero initially causing classification212 // to fail.213 webcamCanvas.width = video.videoWidth;214 webcamCanvas.height = video.videoHeight;215 videoRenderCanvas.width = video.videoWidth;216 videoRenderCanvas.height = video.videoHeight;217 let webcamCanvasCtx = webcamCanvas.getContext(2d);218 webcamCanvasCtx.drawImage(video, 0, 0);219 });220221 video.srcObject = stream;222223 video.addEventListener(loadeddata, predictWebcam);224 });225}226227// We will create a tempory canvas to render to store frames from 228// the web cam stream for classification.229var videoRenderCanvas = document.createElement(canvas);230var videoRenderCanvasCtx = videoRenderCanvas.getContext(2d);231232// Lets create a canvas to render our findings to the DOM.233var webcamCanvas = document.createElement(canvas);234webcamCanvas.setAttribute(class, overlay);235liveView.appendChild(webcamCanvas);236237// If webcam supported, add event listener to button for when user238// wants to activate it.239if (hasGetUserMedia()) {240 const enableWebcamButton = document.getElementById(webcamButton);241 enableWebcamButton.addEventListener(click, enableCam);242} else {243 console.warn(getUserMedia() is not supported by your browser);244}CSS：
1/**2 * @license3 * Copyright 2018 Google LLC. All Rights Reserved.4 * Licensed under the Apache License, Version 2.0 (the "License");5 * you may not use this file except in compliance with the License.6 * You may obtain a copy of the License at7 *8 * http://www.apache.org/licenses/LICENSE-2.09 *10 * Unless required by applicable law or agreed to in writing, software11 * distributed under the License is distributed on an "AS IS" BASIS,12 * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.13 * See the License for the specific language governing permissions and14 * limitations under the License.15 * =============================================================================16 */1718192021/******************************************************22 * Stylesheet by Jason Mayes 2020.23 *24 * Get latest code on my Github:25 * https://github.com/jasonmayes/Real-Time-Person-Removal26 * Got questions? Reach out to me on social:27 * Twitter: @jason_mayes28 * LinkedIn: https://www.linkedin.com/in/creativetech29 *****************************************************/3031323334body {35 font-family: helvetica, arial, sans-serif;36 margin: 2em;37 color: #3D3D3D;38}3940414243h1 {44 font-style: italic;45 color: #FF6F00;46}4748495051h2 {52 clear: both;53}5455565758em {59 font-weight: bold;60}6162636465video {66 clear: both;67 display: block;68}6970717273section {74 opacity: 1;75 transition: opacity 500ms ease-in-out;76}7778798081header, footer {82 clear: both;83}8485868788button {89 z-index: 1000;90 position: relative;91}9293949596.removed {97 display: none;98}99100101102103.invisible {104 opacity: 0.2;105}106107108109110.note {111 font-style: italic;112 font-size: 130%;113}114115116117118.webcam {119 position: relative;120}121122123124125.webcam, .classifyOnClick {126 position: relative;127 float: left;128 width: 48%;129 margin: 2% 1%;130 cursor: pointer;131}132133134135136.webcam p, .classifyOnClick p {137 position: absolute;138 padding: 5px;139 background-color: rgba(255, 111, 0, 0.85);140 color: #FFF;141 border: 1px dashed rgba(255, 255, 255, 0.7);142 z-index: 2;143 font-size: 12px;144}145146147148149.highlighter {150 background: rgba(0, 255, 0, 0.25);151 border: 1px dashed #fff;152 z-index: 1;153 position: absolute;154}155156157158159.classifyOnClick {160 z-index: 0;161 position: relative;162}163164165166167.classifyOnClick canvas, .webcam canvas.overlay {168 opacity: 1;169170 top: 0;171 left: 0;172 z-index: 2;173}174175176177178#liveView {179 transform-origiHtml:
1<!DOCTYPE html>2<html lang="en">3 <head>4 <title>Disappearing People Project</title>5 <meta charset="utf-8">6 <meta http-equiv="X-UA-Compatible" content="IE=edge">7 <meta name="viewport" content="width=device-width, initial-scale=1">8 <meta name="author" content="Jason Mayes">910111213 <!-- Import the webpages stylesheet -->14 <link rel="stylesheet" href="/style.css">1516171819 <!-- Import TensorFlow.js library -->20 <script src="https://cdn.jsdelivr.net/npm/@tensorflow/tfjs/dist/tf.min.js" type="text/javascript"></script>21 </head>22 <body>23 <h1>Disappearing People Project</h1>2425 <header class="note">26 <h2>Removing people from complex backgrounds in real time using TensorFlow.js</h2>27 </header>2829303132 <h2>How to use</h2>33 <p>Please wait for the model to load before trying the demos below at which point they will become visible when ready to use.</p>34 <p>Here is a video of what you can expect to achieve using my custom algorithm. The top is the actual footage, the bottom video is with the real time removal of people working in JavaScript!</p>35 <iframe width="540" height="812" src="https://www.youtube.com/embed/0LqEuc32uTc?controls=0&autoplay=1" frameborder="0" allow="accelerometer; autoplay; encrypted-media; gyroscope; picture-in-picture" allowfullscreen></iframe>3637 <section id="demos" class="invisible">3839404142 <h2>Demo: Webcam live removal</h2>43 <p>Try this out using your webcam. Stand a few feet away from your webcam and start walking around... Watch as you slowly disappear in the bottom preview.</p>4445 <div id="liveView" class="webcam">46 <button id="webcamButton">Enable Webcam</button>47 <video id="webcam" autoplay></video>48 </div>49 </section>505152535455 <!-- Include the Glitch button to show what the webpage is about and56 to make it easier for folks to view source and remix -->57 <div class="glitchButton" style="position:fixed;top:20px;right:20px;"></div>58 <script src="https://button.glitch.me/button.js"></script>5960 <!-- Load the bodypix model to recognize body parts in images -->61 <script src="https://cdn.jsdelivr.net/npm/@tensorflow-models/body-pix@2.0"></script>6263 <!-- Import the pages JavaScript to do some stuff -->64 <script src="/script.js" defer></script>65 </bod66实时演示
你也可以在自己的Web浏览器中根据自己的喜好试着复现一下：
Codepen.io：https://codepen.io/jasonmayes/pen/GRJqgma
Glitch.com：https://glitch.com/~disappearing-people
等待模型加载完成，然后就可以使用了。
这是使用作者自定义算法实现的视频。上半部分是实际镜头，底部是用JavaScript实时删除人物的视频。
用你自己的网络摄像头试一下，要距离摄像头几英尺远，然后来回走动，在底部预览中你会慢慢从画面中消失。赶快试试吧，使用效果别忘了留言和大家一起分享哦！
版权申明
本文系作者 @河马 原创发布在河马博客站点。未经许可，禁止转载。
暂无评论数据