PHP on Coinidea's Blog

[LNMP]Memory Optimization

Wed, 19 Dec 2018 03:01:19 +0000

Problem Description

Recently, my server ran into an issue where it would frequently kill my Tomcat process when memory ran low. Since the overall traffic isn’t very high, LNMP and Tomcat are currently running on the same machine, but this caused Tomcat-related Java applications to become unavailable.

1

free -m

Checking the memory usage, I found the operating system had only about 60MB of memory left – almost completely used up. This situation persisted for about a month, and it was a constant source of frustration. I considered adding more memory but thought it was too expensive. When resources are insufficient, there are generally two approaches: the first is to add resources, and the second is to optimize existing resources. For someone on a budget like me, the second approach is usually my first choice.

Initially, I thought Tomcat itself or some other process was consuming too much memory. After running:

1

top

I found that php-fpm processes were often the ones consuming the most memory.

To view the top 40 memory-consuming processes:

1

ps auxw | head -1; ps auxw | sort -rn -k4 | head -40

I found that php-fpm dominated the top 40 processes, with memory usage ranging from 2% to 7%. The problem was now quite clear – the culprit was php-fpm.

Solution

You can control the number of php-fpm child processes by configuring the pm.max_children property. First, open the php-fpm configuration file:

1

vim /etc/php-fpm.d/www.conf

pm.max_children defaults to 50. Each process uses 1%-2.5% of memory, which adds up to consuming most of the memory. Try reducing the value – I set mine to 25 (also referencing other blogs). At the same time, check the following two properties:

pm.max_spare_servers : This value represents the maximum number of idle processes. If idle processes exceed this value, they will be cleaned up.
pm.min_spare_servers : The minimum number of idle processes. If idle processes fall below this value, new child processes are created.

Neither of these values should exceed the pm.max_children value. Typically, pm.max_spare_servers is set to 60%-80% of the pm.max_children value.

Finally, restart php-fpm:

1

service php-fpm restart

Check the memory again:

1

free -m

I found there was about 600MB of free memory remaining. Even after starting Tomcat, there was still about 300MB to spare.

Reference: https://www.jb51.net/article/129528.htm

1
2

Hope this helps! If you need anything else, feel free to let me know.

[Push Notifications]Push Notifications Based on Umeng Service

Sun, 19 Nov 2017 11:48:10 +0000

When the server interacts with mobile clients, it often needs to push messages from the server to the user’s mobile device. Umeng provides such a push notification service — UPush.

Download the SDK code. Based on the demo, this article provides a simple ThinkPHP integration.

1
2
3
4
5
6


$appkey = "your app key";
$secret = "your app secret";
import('@.ORG.Notification');
$notification = new Notification($appkey, $secret);
$custom_data = array($key=>$value);
$notification->sendIOSBroadcast($your_title, $custom_data);

Notification.class.php is the demo.php from the SDK, encapsulated into ThinkPHP’s ORG folder.
If you have any questions, feel free to leave a comment below. I typically reply once a day.

[ThinkPHP]Table Locking Under Concurrency in ThinkPHP

Fri, 17 Nov 2017 03:10:04 +0000

Recently I needed to build a registration system that doesn’t require very high concurrency, but there’s still a possibility of multiple users registering at the same time. Since registration involves ordering, without row locking or table locking, the following situation can occur.
Example:
Currently the system only has user A registered with order 1, denoted as (A, 1).
At some moment t, users B, C, and D register simultaneously. The backend receives all three requests almost at the same time, queries the registration table, finds only (A, 1), and inserts (B, 2), (C, 2), (D, 2) into the table, resulting in:

1
2
3
4


(A, 1)
(B, 2)
(C, 2)
(D, 2)

My initial idea was: instead of recording the order number, I would record the insertion timestamp, which would give:

1
2
3
4


(A, ta)
(B, tb)
(C, tc)
(D, td)

ta, tb, tc, td are almost impossible to be equal.
However, this approach has three unsolvable issues:

There is a (very low) probability that timestamps could be equal
It’s not intuitive and requires sorting by timestamp
Registration has a capacity limit (e.g., 40 spots) and needs to immediately report whether the registration was successful

The current solution is to use database locking.
After researching online, ThinkPHP supports both pessimistic and optimistic locking. Given the low traffic of the target system, pessimistic locking is sufficient.

MyISAM can only lock tables, while InnoDB supports row-level locking. Table locking is sufficient for the target system.
The official documentation provides this table locking approach:

1

$User->lock(true)->save($data);// Use pessimistic locking

However, the target system needs to perform a series of operations, so the table locking code used is:

1
2
3
4


M()->query("lock tables yourtable write");
// TODO
// your code
M()->query("unlock tables");

The actual results worked quite well [real dates have been redacted]:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40


??-??-?? 10:00:00 1
??-??-?? 10:00:00 2
??-??-?? 10:00:00 3
??-??-?? 10:00:00 4
??-??-?? 10:00:01 5
??-??-?? 10:00:01 6
??-??-?? 10:00:01 7
??-??-?? 10:00:01 8
??-??-?? 10:00:02 9
??-??-?? 10:00:02 10
??-??-?? 10:00:02 11
??-??-?? 10:00:02 12
??-??-?? 10:00:02 13
??-??-?? 10:00:02 14
??-??-?? 10:00:03 15
??-??-?? 10:00:03 16
??-??-?? 10:00:03 17
??-??-?? 10:00:05 18
??-??-?? 10:00:06 19
??-??-?? 10:00:07 20
??-??-?? 10:00:08 21
??-??-?? 10:00:10 22
??-??-?? 10:00:15 23
??-??-?? 10:00:17 24
??-??-?? 10:00:19 25
??-??-?? 10:00:19 26
??-??-?? 10:00:24 27
??-??-?? 10:00:25 28
??-??-?? 10:00:34 29
??-??-?? 10:00:35 30
??-??-?? 10:00:38 31
??-??-?? 10:01:06 32
??-??-?? 10:01:11 33
??-??-?? 10:01:11 34
??-??-?? 10:01:17 35
??-??-?? 10:01:18 36
??-??-?? 10:02:27 37
??-??-?? 10:02:38 38
??-??-?? 10:02:39 39
??-??-?? 10:02:57 40

[PHP] Filtering All Punctuation with PHP Regular Expressions

Thu, 08 Dec 2016 09:40:15 +0000

 1
 2
 3
 4
 5
 6
 7
 8
 9
10


// Filter English punctuation
$word = preg_replace("/[[:punct:]]/i", " ", $word);

// Filter Chinese punctuation
mb_regex_encoding('utf-8');
$char = "。、！？：；﹑•＂…''""〝〞∕¦‖—　〈〉﹞﹝「」‹›〖〗】【»«』『〕〔》《﹐¸﹕︰﹔！¡？¿﹖﹌﹏﹋＇´ˊˋ―﹫︳︴¯＿￣﹢﹦﹤‐­˜﹟﹩﹠﹪﹡﹨﹍﹉﹎﹊ˇ︵︶︷︸︹︿﹀︺︽︾ˉ﹁﹂﹃﹄︻︼（）";
$word = mb_ereg_replace("[".$char."]", " ", $word, "UTF-8");

// Filter consecutive spaces
$word = preg_replace("/\s+/", " ", $word);

mb_ereg_replace is used for Chinese character replacement. When using regex, you don’t need to add / / delimiters.

[PHP] Large-Scale Data Import and Export with ThinkPHP, PHP, and MySQL

Thu, 08 Dec 2016 09:37:45 +0000

Large-Scale Data Import and Export

Note: “Large-scale” here refers to importing and exporting 100,000+ records from text files (including Office documents such as docx, xlsx).

MySQL has two very common table types: MyISAM and InnoDB. There are many comparisons of these two types online. The main difference is that InnoDB supports transactions and row-level locking, while MyISAM does not. MyISAM is relatively more efficient, and each has its own advantages and disadvantages.

Use Cases for MyISAM and InnoDB

MyISAM is suitable for:

Frequent COUNT calculations;
Infrequent inserts but frequent queries;
No transactions needed.

InnoDB is suitable for:

High reliability requirements or transaction support needed;
Frequent table updates and queries with high likelihood of table locking.

Differences Between MyISAM and InnoDB

MyISAM does not support advanced features like transactions, while InnoDB does.
MyISAM tables do not support foreign keys.
During write operations (insert, update, delete), MyISAM locks the entire table, while InnoDB locks only the affected rows.
When your database has heavy writes, updates, and deletions with relatively few queries, or when data integrity requirements are high, choose InnoDB. When your database is primarily read-heavy with relatively few updates and writes, and data integrity requirements are less strict, choose MyISAM, as its query performance is faster than InnoDB.

In summary, InnoDB is a good choice for this scenario.

Optimization Methods for Inserting Large Amounts of Data

When PHP needs to insert large amounts of data, inserting one record at a time results in severe I/O performance bottlenecks.

Demo Code:

1
2
3
4
5
6
7


$params = array('value' => '50');
set_time_limit(0);
echo date("H:i:s");
for($i = 0; $i < 2000000; $i++) {
 $connect_mysql->insert($params);
}
echo date("H:i:s");

There are two methods to solve this problem:

1. Using Transactions (InnoDB supported)

For example, you can commit a transaction every 10,000 or 100,000 records, which greatly reduces I/O.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12


echo date("H:i:s");
$connect_mysql->query('BEGIN');
$params = array('value' => '50');
for($i = 0; $i < 2000000; $i++) {
 $connect_mysql->insert($params);
 if($i % 100000 == 0) {
 $connect_mysql->query('COMMIT');
 $connect_mysql->query('BEGIN');
 }
}
$connect_mysql->query('COMMIT');
echo date("H:i:s");

2. SQL Statement Concatenation

This is a MySQL feature that uses optimized SQL statements: concatenate SQL statements using insert into table () values (),(),(),() and insert them all at once. If the string is too long, you need to configure MySQL by running:

1

SET GLOBAL max_allowed_packet = 2 * 1024 * 1024 * 10;

1
2
3
4
5
6


$sql = "INSERT INTO twenty_million (value) VALUES";
for($i = 0; $i < 2000000; $i++) {
 $sql .= "('50'),";
}
$sql = substr($sql, 0, strlen($sql) - 1);
$connect_mysql->query($sql);

The first method is more commonly used, while the second is mostly for test data.

By default, PHP has a memory limit of 128MB. When ThinkPHP exceeds this memory limit, you can either increase PHP’s default maximum memory or simply remove the APP_DEBUG constant definition in the entry file or set it to false. This will save a lot of memory. Of course, you can also use transaction mode to commit every n records. ThinkPHP’s addAll function can also be used for batch inserts, though it has quite a few bugs.

Starting and Committing Transactions in ThinkPHP

1
2


$table->startTrans();
$table->commit();

When working with Office files, PHPWord and PHPExcel are recommended.

[PHP] call to undefined function mb_convert_encoding

Tue, 02 Aug 2016 03:37:01 +0000

Solution on Windows

In php.ini, enable the mbstring library: remove the semicolon before extension=php_mbstring.dll.

Solution on Linux

First, install the php-mbstring library:
1

yum install php-mbstring
Then, edit the /etc/php.ini file and add the following line:
1

extension=mbstring.so
Save and restart the httpd service:
1

service httpd restart

[Memcache] Testing Memcache Connection with PHP

Tue, 08 Mar 2016 12:42:07 +0000

1
2
3
4
5


<?php
$memcache_obj = memcache_connect('127.0.0.1', 11211);
$memcache_obj->add("name", "test");
echo $memcache_obj->get("name");
?>

If you encounter issues using this code, make sure of the following:

Is the Memcache service running: Ensure your Memcache service is running on 127.0.0.1 at port 11211.
Is the PHP Memcache extension installed and enabled: Make sure you have installed and enabled the PHP Memcache extension.

You can install the Memcache extension with the following command:

1

sudo apt-get install php-memcache

And enable the extension in your php.ini file:

1

extension=memcache.so

Restart your web server to apply changes:

1

sudo service apache2 restart

Or, if you’re using Nginx:

1

sudo service nginx restart

If you’re using memcached instead of memcache, make sure to use the correct extension and functions:

1
2
3
4
5
6


<?php
$memcached = new Memcached();
$memcached->addServer('127.0.0.1', 11211);
$memcached->set("name", "test");
echo $memcached->get("name");
?>

In this case, you need to install the php-memcached extension:

1

sudo apt-get install php-memcached

[PHP] Encrypting PHP with ZendGuard

Tue, 29 Dec 2015 07:28:22 +0000

Downloading ZendGuard

Official download link: http://www.zend.com/en/products/guard/downloads#Windows

Registration is required before downloading. I downloaded Zend Guard 6.0. Simply double-click to run the installer.

The latest version is 7.0. If you want to download an earlier version, click “>Download here” and a list of older versions will appear below the current page.

Downloading PHP

Official download links: http://php.net/downloads.php | http://php.net/releases/

My current version: http://windows.php.net/download/#php-5.4

PHP typically releases two types of builds for each version: Thread Safe and Non Thread Safe.

Important note: ZendGuard only supports Non Thread Safe builds. So please download the Non Thread Safe version of PHP.

Configuring PHP – ZendLoader.dll

Download ZendLoader.dll

Official download link: http://www.zend.com/en/products/loader/downloads#Windows

Registration is required before downloading. Similar to ZendGuard, make sure to select the ZendLoader version that matches your PHP version.

Copy ZendLoader.dll to the PHP [PHP root directory]\ext\ folder.

Rename php.ini-development or php.ini-production to php.ini,
and add the following content to php.ini:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13


zend_extension="./ext/ZendLoader.dll"

; Enables loading encoded scripts. The default value is On
zend_loader.enable=1

; Disable license checks (for performance reasons)
zend_loader.disable_licensing=0

; The Obfuscation level supported by Zend Guard Loader. The levels are detailed in the official Zend Guard Documentation. 0 - no obfuscation is enabled
zend_loader.obfuscation_level_support=3

; Path to where licensed Zend products should look for the product license. For more information on how to create a license file, see the Zend Guard User Guide
zend_loader.license_path=

Creating PHP Code

Create an input folder, and inside it create test.php.

Code:

1
2
3
4
5


<?php
date_default_timezone_set("Asia/shanghai");
echo "Thanks ZendGuard!\n";
echo "Now datetime is ".date("Y-m-d h:i:s")."\n";
?>

The above code outputs:

Thanks ZendGuard

Now datetime is [current time].

Installing ZendGuard and Creating a Project

ZendGuard has a straightforward installer. After installation, launch it and you’ll see:

Create a new Zend Guard Project.
Set the output directory to the output folder.
Add the input folder as the input directory.
Select the PHP version.
Click
A new test.php is generated in the output folder. Opening it reveals garbled content (the encrypted code).

Running Source Code and Encrypted Code

Running the source code:

Running the encrypted code:

With this, you can now use ZendGuard to encrypt PHP code.

[PHP] Configuring PHPUnit for ThinkPHP

Thu, 10 Dec 2015 14:19:57 +0000

Version Details

PHP: 5.3

PHPUnit

ThinkPHP 3.1.3

IDE: PHPStorm 10 (recommended)

Configuring PHPUnit in PHPStorm

See the blog post:

http://blog.coinidea.com/web开发/php-1088.html

ThinkPHP Deployment

Official code download:

http://www.thinkphp.cn/down.html

Site initialization:

http://www.thinkphp.cn/info/60.html

Test Cases

In this example, the index.php in the root directory is configured as follows:

1
2
3
4
5
6
7


<?php
define('APP_NAME', 'example');
define('APP_PATH', '../example/');
define('APP_PHPUNIT', false);
define('APP_DEBUG', true);
require('../ThinkPHP/ThinkPHP.php');
?>

After the first visit, the following directory structure is generated:

Create a new folder named “Testcase” in the example site.

Testing the Model

Create HelloModel.class.php:

1
2
3
4
5
6
7
8
9


<?php
class HelloModel extends Model
{
 public function sayHello()
 {
 print 'Hello';
 return 'Hello';
 }
}

Create a Test.php file in the Test folder as the PHPUnit test file. Note that you need to require ThinkPHP to initialize the framework environment. Also, in Think.class.php, modify the

start() function by changing App::run() to !APP_PHPUNIT && App::run();

This distinguishes between site runtime and test case execution.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12


<?php
define('APP_NAME', 'example');
define('APP_PATH', './../../example/');
define('APP_PHPUNIT', true);
require('./../../ThinkPHP/ThinkPHP.php');
class TestSayHello extends PHPUnit_Framework_TestCase {

public function setUp() { }

public function tearDown(){ }

}

Add a test case in TestSayHello:

1
2
3
4
5


public function testHelloModel()
{
 $hello = D('Hello');
 $this->assertTrue($hello->sayHello('Hello') == 'Hello');
}

Testing the Action

Modify IndexAction.class.php as follows:

1
2
3
4
5
6
7
8
9


<?php
class IndexAction extends Action
{
 public function index()
 {
 $hello = D("Hello");
 return $hello->sayHello();
 }
}

Browser access to Index:

Add a test case in TestSayHello:

1
2
3
4
5


public function testHelloAction()
{
 $hello = new IndexAction();
 $this->assertTrue($hello->index() == 'Hello');
}

Results

Running Test.php produces the following result:

Tests passed. At this point, unit testing has been successfully added to ThinkPHP.

poisonbian 2016/05/10 14:57

I tried following this guide and found that the model couldn’t be used, and custom functions in the Common directory were not loaded. After some investigation, I think adding the APP_PHPUNIT check in Think.class.php is not ideal. It would be better to add it in App.class.php instead: !APP_PHPUNIT && App::exec();

[PHP] PHPUnit Installation, Configuration, and Examples

Fri, 04 Dec 2015 14:45:19 +0000

PHPUnit Installation and Sample Test

Many people online recommend installing via PEAR. I tried various methods from different blog posts, but all failed.
The method that worked for me is more straightforward:

Download link:

https://phpunit.de/index.html

The download is a .phar PHP extension. Download the PHPUnit version that matches your PHP version.

Follow the instructions at https://phpunit.de/manual/current/en/installation.html#installation.phar.windows step by step.

Download PHPUnit.phar to a folder, for example: C:\bin\
Add this folder C:\bin\ to the Path environment variable
In the command line, navigate to C:\bin and enter:
echo @php "%~dpophpunit.phar"%*>phpunit.cmd

In any command line location, enter:

1

phpunit --version

You should see:

At this point, PHPUnit is successfully installed.

Test Example:

SayHello.php

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11


<?php

class SayHello
{
 public function printHello()
 {
 echo 'Hello';
 return 'Hello';
 }
}
?>

TestSayHello.php

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17


<?php

require_once 'SayHello.php';

class TestSayHello extends PHPUnit_Framework_TestCase
{
 public function setUp() { }

 public function tearDown() { }

 public function testConnectionIsValid()
 {
 $hi = new SayHello();
 $this->assertTrue($hi->printHello() == 'Hello');
 }
}
?>

The SayHello class prints “Hello” to the screen and returns “Hello”.

TestSayHello verifies whether the SayHello class correctly returns the “Hello” string.

Run it in the command line, and the test passes.

Configuring PHPUnit in PHPStorm

Older versions of PHPStorm may not support PHPUnit well and will throw:

Warning: require_once(PHPUnit/Runner/Version.php)

See:

http://stackoverflow.com/questions/30278588/phpstorm-via-phpunit-phar-warning-require-oncephpunit-runner-version-php

This article uses PHPStorm 10. Download link:

https://www.jetbrains.com/phpstorm/?fromMenu

Open the two files from earlier

Open Settings: File => Settings

Configure PHP version:

Configure PHPUnit

Sample Test

Select TestSayHello and run it directly. The result:

[Yii] Defending Against CSRF and XSS in Yii [Part 2]

Tue, 21 Jul 2015 16:04:21 +0000

I previously wrote Defending Against CSRF and XSS in Yii [Part 1], but when it came to actual testing, there was a requirement to filter out special characters such as :, :, “, <, >, %, etc.

Yii’s CHtml::purifier actually only filters HTML, and CHtml::encode essentially calls htmlspecialchars:

htmlspecialchars only converts <, >, single quotes, double quotes, and &
htmlentities converts all HTML entities

As you can see, PHP’s built-in functions cannot filter all special characters. The approach I took was to create a new Filter class within the Yii framework and write a custom specialchar function, allowing you to replace characters as needed. For better performance, it can be defined as public static.

1
2
3
4
5
6
7


class Filter {
 public static function ReplaceSpecialChar($str) {
 $str=str_replace(":",$str); //TODO: replace other chars
 $str = htmlspecialchars($str);
 return $str;
 }
}

[Yii] Defending Against CSRF and XSS in Yii [Part 1]

Sun, 19 Jul 2015 02:59:44 +0000

CSRF stands for Cross-Site Request Forgery, and XSS stands for Cross-Site Scripting. To be honest, I only recently encountered these for the first time. After building a website with Yii and scanning it with testing software (e.g., IBM AppScan), two relatively common and serious issues were found: CSRF and XSS. So how do we fix them?

– CSRF

Reference: http://www.crarun.com/article-7.html

Yii supports CSRF configuration in its config file. Once configured, it embeds a hidden token with a hash key value in forms, and the server verifies whether each request is authorized through this token.

1
2
3
4
5
6


'components'=>array(
 'request'=>array(
 // Enable Yii Validate CSRF Token
 'enableCsrfValidation' => true,
 ),
),

In practice, because I was using an older version of Yii, the framework didn’t automatically create the input element in forms, so I had no choice but to create it manually by inserting the following code:

1

<input type="hidden" value="<?php echo Yii::app()->getRequest()->getCsrfToken(); ?>" name="YII_CSRF_TOKEN" />

Another issue arose: since CSRF wasn’t considered thoroughly early on, the entire system contained numerous AJAX POST requests, meaning lots of JS code had POST submissions. However, Yii’s view rendering doesn’t affect code inside .js files - even if you add the above code in a .js file, PHP won’t produce any output. So I came up with a workaround:

Add the following code to JS POST submissions:

1

$("input[name='YII_CSRF_TOKEN']").val()

Then add the hidden input in the PHP view. However, this also reveals a limitation - it doesn’t fully prevent CSRF. The optimal solution would be to embed authorization in each user’s session.

– XSS

When searching for solutions to this problem, a common question is: at which stage should string filtering be done - submission, storage, or display? I saw an answer on SegmentFault that said both are essential: one prevents malicious data from entering the database, and the other prevents it from being executed.

In my opinion, front-end user validation during submission is essential for good user experience. However, server-side validation is still a must, since programs can bypass the front end.

Should we also filter during display? I recommend yes - it adds an extra layer of safety to prevent attacks from taking effect.

How to do it? Yii provides CHtml and CHTMLPurifier for filtering. On GitHub, many people have open-sourced good front-end libraries for filtering - just search for them.

[Redis] Configuring Redis in Yii - Common Issues

Fri, 12 Jun 2015 06:24:58 +0000

Basic configuration:

In Yii’s main.php:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13


return array(
 ...
 'components'=>array(
 'redis_cache' => array (
 'class' => 'system.caching.CRedisCache',
 'hostname' =>'127.0.0.1',
 'port' => 6379,
 'password'=>'123456',
 'database'=>1
 ),
 ),
 ...
)

Notes:
The CRedisCache in the class field is the official Redis plugin;
The password field is required if a password has been set;
The database field specifies the corresponding database.

Usage:
$r_key = “key”;
Yii::app()->redis_cache->set($r_key, 99999);
echo Yii::app()->redis_cache->get($r_key);

However, you won’t find a key named “key” in the Redis database, because Yii’s Redis plugin applies MD5 hashing to keys by default.
By examining CCache, the parent class of CRedisCache, you’ll find that you need to declare the following two variables in CRedisCache.php:

1
2


public $hashKey = false;
public $keyPrefix = "";

This solves the problem.

Additionally, if you encounter garbled Chinese characters, it’s because Redis defaults to ANSI encoding. When connecting, use the following command:
./redis-cli –raw -h 127.0.0.1
to connect.

For official documentation, see: http://www.yiiframework.com/doc/api/1.1/CRedisCache/.

Online QR Code Generator

Sat, 30 May 2015 03:19:13 +0000

Sharing a simple online QR code generator I wrote. Click the link below: