Cesium Chinese Website: http://cesiumcn.org/ | China Fast Access: http://cesium.coinidea.com/

One of the topics from friends of the Cesium Chinese Website is: independently developing a map (tile) download tool.

This topic will be presented in multiple installments as blog posts [for all users] + videos [for paid users]. The source code will be continuously committed and updated. GitHub address: https://github.com/hujiulin/MapDownloader. The source code is open to all users. If you find it helpful, please give it a star for encouragement.

The mightiest sword has no edge; the greatest skill appears effortless.

The tool will ultimately be presented in C# and JavaScript. Programming languages are just the form – understanding the internal logic and workflow allows you to implement it in Java, Python, PHP, or any other language.

The currently open-sourced tool is very simple (bare-bones), but I will regularly update and maintain the code. If you have any questions, you can submit issues on GitHub, or leave messages and ask questions on the WeChat Official Account: Cesium Chinese Website; QQ Group: 807482793; Forum: http://cesium.coinidea.com/.

Prerequisites

Find a programming language you’re familiar with that supports:

1. File download over the network.
2. File I/O, reading and writing local folders and files.
3. Multithreading.
4. Desktop windows, preferably with a browser control.

Generally speaking, modern programming languages support most of the above features. This article uses C#. In the middle part of this series, JavaScript will be introduced.

Tile Map Server

This article uses Baidu Maps. Most tile map servers have x, y, z (level) parameters. The Baidu Maps URL pattern is: http://online{0}.map.bdimg.com/onlinelabel/?qt=tile&x={1}&y={2}&z={3}&styles=pl&udt=20200727&scaler=1&p=0

If we set x=1, y=1, z=3, and open the URL in a browser:

http://online0.map.bdimg.com/onlinelabel/?qt=tile&x=1&y=1&z=3&styles=pl&udt=20200727&scaler=1&p=0

Sample tile

Tile Download

Now that we know the tile generation rules, we just need to specify the download URL to download the corresponding image. The core download code in C# is straightforward and can be easily found online:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26

    private static WebClient wc = null;
    private static void initWebClientInstance()
    {
        if (null == wc)
        {
            wc = new System.Net.WebClient();
        }
    }
    private static void download(string url, string filename)
    {
        // Check filename exist or not
        if (File.Exists(filename))
        {
            return;
        }
    
        // Create directory
        string dir = Path.GetDirectoryName(filename);
        if (!Directory.Exists(dir))
        {
            Directory.CreateDirectory(dir);
        }
    
        initWebClientInstance();
        wc.DownloadFile(url, filename);
    }

Batch Tile Download

Next, we need to:

1. Generate download URLs in bulk.
2. Download each tile in a loop.

As mentioned earlier, the core concept of maps is a quadtree, so we can simply assume that the tile map is also composed of a quadtree, with approximately 18 levels (verifiable online). Thus:

Quadtree

The core batch download code is as follows:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14

    public static void downloadBatch(string urlPattern, string targetFoler, int level)
    {
        int maxX = (int)Math.Pow(2, level - 1);
        int maxY = (int)Math.Pow(2, level - 1);
        for (int x = 0; x <= maxX; x++)
        {
            for (int y = 0; y <= maxY; y++)
            {
                string url = String.Format(urlPattern, 1, x, y, level);
                string filename = Path.Combine(targetFoler, level.ToString(), x.ToString()) + "\\" + y.ToString() + ".png";
                download(url, filename);
            }
        }
    }

The above code will work for downloading, but there are many issues. These will be discussed in the next installment.

Current software interface

Download results

Next Steps

1. The current code only works for small zoom levels. At higher levels, the number of images to download becomes enormous. Multithreading, or even multiple machines, will be needed.
2. Many of the downloaded images are pure black, which suggests our generation rules need improvement.
3. The browser control and the software are not yet linked – you can’t select a region in real-time for downloading.
4. Later, image stitching may be needed.
5. When loading into specific platforms, coordinate system transformations may be required.
6. Automatic control of download intervals, dynamic IP switching.

These issues will be discussed in subsequent articles.

Disclaimer

Generally speaking, map servers require significant storage and bandwidth resources. This article discusses the internal principles of current download tools from an academic perspective only. Corrections are welcome.

Again: GitHub address: https://github.com/hujiulin/MapDownloader. The source code is open to all users. If you find it helpful, please give it a star for encouragement.

This article and the software are for academic exchange only. Commercial use is strictly prohibited.

Cesium Chinese Website QQ Group: 807482793

Cesium Chinese Website: http://cesiumcn.org/ | China Fast Access: http://cesium.coinidea.com/