[Node.js] Building an Image Server with UEditor + Node.js + SeaweedFS

Wed, 18 May 2016 03:16:01 +0000

1. UEditor + Node.js Image Upload

UEditor is a feature-rich open-source rich text editor by Baidu.
Download link: http://ueditor.baidu.com/website/download.html
It currently provides versions for PHP, ASP, .Net, and JSP. UEditor is primarily built with frontend HTML, CSS, and JS. The reason it is further divided by server-side language is, in my understanding, mainly for the image upload functionality which involves server interaction.
Among the many versions, there is no Node.js version. Below I will explain how to convert the PHP version of UEditor into a Node.js version.
After examining all requests in the PHP version, I found that the action parameter values include config (configuration file), uploadimage (image upload), listimage (online management), and catchimage (image capture). So we just need to rewrite these 4 request handlers to meet our needs.

1.1 Modify the `serverUrl` property in UEditor’s `ueditor.config.js`:

1

serverUrl: '/ue/uploads'

1.2 Rename `ueditor/php/config.json` to `config.js` and move it to the `ueditor` directory.

1.3 Next, write the four corresponding actions on the Node.js side. The image upload uses the `connect-busboy` middleware. Code as follows:

 1
 2
 3
 4
 5
 6
 7
 8
 9
 10
 11
 12
 13
 14
 15
 16
 17
 18
 19
 20
 21
 22
 23
 24
 25
 26
 27
 28
 29
 30
 31
 32
 33
 34
 35
 36
 37
 38
 39
 40
 41
 42
 43
 44
 45
 46
 47
 48
 49
 50
 51
 52
 53
 54
 55
 56
 57
 58
 59
 60
 61
 62
 63
 64
 65
 66
 67
 68
 69
 70
 71
 72
 73
 74
 75
 76
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104


// Image upload
var path = require('path');
var uploadsPath = path.resolve('public/uploads') + '/'; // Path to store images
var busboy = require('connect-busboy');
app.use(busboy({
 limits: {
 fileSize: 10 * 1024 * 1024 // 10MB
 }
}));
var action = {
 // Upload image
 uploadimage: function (req, res) {
 var fstream;
 req.pipe(req.busboy);
 req.busboy.on('file', function (fieldname, file, filename, encoding, mimetype) {
 var filesize = 0;
 var ext = path.extname(filename);
 var newFilename = (new Date() - 0) + ext;
 fstream = fs.createWriteStream(uploadsPath + newFilename);
 file.on('data', function (data) {
 filesize = data.length;
 });
 fstream.on('close', function () {
 console.log(JSON.stringify({
 "originalName": filename,
 "name": newFilename,
 "url": '/uploads/' + newFilename,
 "type": ext,
 "size": filesize,
 "state": "SUCCESS"
 }));
 res.send(JSON.stringify({
 "originalName": filename,
 "name": newFilename,
 "url": '/uploads/' + newFilename,
 "type": ext,
 "size": filesize,
 "state": "SUCCESS"
 }));
 });
 file.pipe(fstream);
 });
 },
 // Get configuration file
 config: function (req, res) {
 return res.redirect('/js/UEditor/config.js');
 },
 // Online management
 listimage: function (req, res) {
 fs.readdir(uploadsPath, function (err, files) {
 var total = 0, list = [];
 files.sort().splice(req.query.start, req.query.size).forEach(function (a, b) {
 /^.+\..+$/.test(a) &&
 list.push({
 url: '/uploads/' + a,
 mtime: new Date(fs.statSync(uploadsPath + a).mtime).getTime()
 });
 });
 total = list.length;
 res.json({state: total === 0 ? 'no match file' : 'SUCCESS', list: list, total: total, start: req.query.start});
 });
 },
 // Catch image (save image to server when pasting)
 catchimage: function (req, res) {
 var list = [];
 req.body.source.forEach(function (src, index) {
 http.get(src, function (_res) {
 var imagedata = '';
 _res.setEncoding('binary');
 _res.on('data', function (chunk) {
 imagedata += chunk
 });
 _res.on('end', function () {
 var pathname = url.parse(src).pathname;
 var original = pathname.match(/[^/]+\.\w+$/g)[0];
 var suffix = original.match(/[^\.]+$/)[0];
 var filename = Date.now() + '.' + suffix;
 var filepath = uploadsPath + 'catchimages/' + filename;
 fs.writeFile(filepath, imagedata, 'binary', function (err) {
 list.push({
 original: original,
 source: src,
 state: err ? "ERROR" : "SUCCESS",
 title: filename,
 url: '/uploads/catchimages/' + filename
 });
 })
 });
 })
 });
 var f = setInterval(function () {
 if (req.body.source.length === list.length) {
 clearInterval(f);
 res.json({state: "SUCCESS", list: list});
 }
 }, 50);
 }
};
app.get('/ue/uploads', function (req, res) {
 action[req.query.action](req, res);
});
app.post('/ue/uploads', function (req, res) {
 action[req.query.action](req, res);
});

The above mainly references this blog: http://www.xiaoboy.com/detail/1341545081.html

2. GoLang Installation and Configuration

In section 1, UEditor uploads images to the /public/uploads/ folder on the Node.js server. If you need an image server with high storage, portability, and scalability, you need to set up a dedicated server. This article uses the open-source distributed image (file) server seaweedfs.
GitHub link: https://github.com/chrislusf/seaweedfs
This server is written in GoLang, so you need to install and configure GoLang. Download link for China: http://www.golangtc.com/download – choose the appropriate package for your operating system. This article demonstrates installation on Windows 7 x64. After a straightforward installation, you need to configure the following environment variables:

1
2
3


GOROOT=D:\Go16
GOPATH=D:\ImageServer\seaweedfs
PATH=D:\Go16\bin

The GOPATH environment variable is very important – your own code must be placed in the directory configured by this variable for the Go compiler to find and compile it.

3. Compiling and Running seaweedfs

Clone the seaweedfs code from GitHub, then follow the makefile instructions or run the makefile directly. In the seaweedfs directory, run the following commands in the console:

1
2
3
4
5
6


go clean -i -v ./go/weed/

rm -f weed #for linux

go get -v -d ./go/weed
go build -v -o weed ./go/weed

Some dependencies may fail to download during go get and need to be downloaded manually and placed in the \seaweedfs\src directory.
Here is a recommended Go package download site: https://gopm.io/download?pkgname=golang.org/x/net
After building, a weed binary will be generated in the seaweedfs directory. You can also rename it to weed.exe. Now you can start weed. Using the local machine as an example:

1

weed master

1

weed volume --dir="./tmp/data1" --max=5 --mserver="localhost:9333" --port=8080 &

Following the GitHub instructions, uploading and downloading images works without issues. At this point, the basic image server setup is complete.

Modify the Node.js image upload code as follows:

 1
 2
 3
 4
 5
 6
 7
 8
 9
 10
 11
 12
 13
 14
 15
 16
 17
 18
 19
 20
 21
 22
 23
 24
 25
 26
 27
 28
 29
 30
 31
 32
 33
 34
 35
 36
 37
 38
 39
 40
 41
 42
 43
 44
 45
 46
 47
 48
 49
 50
 51
 52
 53
 54
 55
 56
 57
 58
 59
 60
 61
 62
 63
 64
 65
 66
 67
 68
 69
 70
 71
 72
 73
 74
 75
 76
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
111
112
113


// Image upload
var path = require('path');
var multipart = require('connect-multiparty');
var multipartMiddleware = multipart();
function curlPostAssign(res, filepath, filename, filesize) {
 request.post(config.url_image + 'dir/assign',{},
 function (error, response, body) {
 if (!error && response.statusCode == 200) {
 assign = eval('(' + body + ')');
 var result = {};
 result.res = res;
 result.json = JSON.stringify({
 "originalName": filename,
 "name": assign.fid,
 "url": '/res?fid=' + assign.fid,
 "type": ext,
 "size": filesize,
 "state": "SUCCESS"
 });
 curlPostWrite(assign.fid, filepath, assign.publicUrl, result);
 var ext = path.extname(filename);
 } else {
 console.log("Image server assign error...")
 }
 });
}
function curlPostWrite(fid, tmp_file, publicUrl, result) {
 var files = [
 {urlKey: "file1", urlValue: tmp_file}
 ]
 var options = {
 host: publicUrl.split(":")[0],
 port: publicUrl.split(":")[1],
 method: "POST",
 path: "/" + fid
 }
 var req = http.request(options, function(res){
 res.on("data", function(chunk){
 })
 })

 req.on('error', function(e){
 console.log('problem with request:' + e.message);
 console.log(e);
 })
 postfile.postFile(files, req, result);
}
var action = {
 // Upload image
 uploadimage: function(req, res) {
 curlPostAssign(res, req.files.upfile.path, req.files.upfile.originalFilename, req.files.upfile.size);
 },
 // Get configuration file
 config: function (req, res) {
 return res.redirect('/js/UEditor/config.js');
 },
 // Online management
 listimage: function (req, res) {
 fs.readdir(uploadsPath, function (err, files) {
 var total = 0, list = [];
 files.sort().splice(req.query.start, req.query.size).forEach(function (a, b) {
 /^.+\..+$/.test(a) &&
 list.push({
 url: '/uploads/' + a,
 mtime: new Date(fs.statSync(uploadsPath + a).mtime).getTime()
 });
 });
 total = list.length;
 res.json({state: total === 0 ? 'no match file' : 'SUCCESS', list: list, total: total, start: req.query.start});
 });
 },
 // Catch image (save image to server when pasting)
 catchimage: function (req, res) {
 var list = [];
 req.body.source.forEach(function (src, index) {
 http.get(src, function (_res) {
 var imagedata = '';
 _res.setEncoding('binary');
 _res.on('data', function (chunk) {
 imagedata += chunk
 });
 _res.on('end', function () {
 var pathname = url.parse(src).pathname;
 var original = pathname.match(/[^/]+\.\w+$/g)[0];
 var suffix = original.match(/[^\.]+$/)[0];
 var filename = Date.now() + '.' + suffix;
 var filepath = uploadsPath + 'catchimages/' + filename;
 fs.writeFile(filepath, imagedata, 'binary', function (err) {
 list.push({
 original: original,
 source: src,
 state: err ? "ERROR" : "SUCCESS",
 title: filename,
 url: '/uploads/catchimages/' + filename
 });
 })
 });
 })
 });
 var f = setInterval(function () {
 if (req.body.source.length === list.length) {
 clearInterval(f);
 res.json({state: "SUCCESS", list: list});
 }
 }, 50);
 }
};
app.get('/ue/uploads', multipartMiddleware, function (req, res) {
 action[req.query.action](req, res);
});
app.post('/ue/uploads', multipartMiddleware, function (req, res) {
 action[req.query.action](req, res);
});

Running result:

[Node.js] Exception Handling in Node.js Express

Thu, 10 Mar 2016 09:30:18 +0000

Since Node.js is non-blocking, single-process, and single-threaded, once an exception is thrown, the entire service stops, making it very unstable. Here are the solutions:

Make the program robust: Wrap all potentially exception-prone areas with try { } catch(){ }.
Express error handling mechanism: As a commonly used Node.js framework, Express has its own error handling mechanism.

1
2
3
4
5
6


// Express' errorHandler
function errorHandler(err, req, res, next) {
 console.error(err.stack);
 res.status(500).send('Something broke!');
}
app.use(errorHandler);

Use the domain module: First install the domain module with npm install domain

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15


// Domain exception handling
const domain = require('domain');

app.use(function(req, res, next) {
 var reqDomain = domain.create();
 reqDomain.on('error', function(err) {
 console.log(err.stack);
 res.statusCode = 500;
 res.end(err.message + '\n');
 reqDomain.dispose();
 });
 reqDomain.add(req);
 reqDomain.add(res);
 reqDomain.run(next);
});

Use forever to start app.js: Forever acts as a daemon process for Node.js that can start, stop, and restart your app.

1

forever start app.js

Through these methods, you can improve the stability of Node.js services and prevent the entire service from crashing due to uncaught exceptions.

Nodejs on Coinidea's Blog