16 December 2013

Amazon S3 vs Google Cloud Storage

I was thinking to move the thumbnails of alwayshere.net to either google cloud storage or to amazon s3. Pricing is more or less the same for the traffic/storage I need, so it's all about speed in my case. The detailed data from the tables was generated using the following command:
curl -s -w "Total: "%{time_total}" ; DNS lookup: "%{time_namelookup}" ; Connect time: "%{time_connect}" ; Pretransfer time: "%{time_pretransfer}" ; Start transfer: "%{time_starttransfer} -o /dev/null URL
Both, google cloud storage and amazon s3, were tested with HTTPS requests.

Below you can find the results, first the averages, then the detailed / region tested. Sorry for the bad formatting, didn't have time to make it look pretty.

AVERAGESGoogle Cloud StorageAmazon S3
Total time:1.1682.374
DNS lookup time:0.4790.911
Connection time:0.6081.071
Pretransfer time:0.7131.400
Start transfer time:0.9931.601


Taiwan, ASIAGoogle Cloud StorageAmazon S3
Total time:0.5475.053
DNS lookup time:0.0360.008
Connection time:0.0420.296
Pretransfer time:0.0630.886
Start transfer time:0.4371.198


Czech Republic, EUGoogle Cloud StorageAmazon S3
Total time:0.3370.512
DNS lookup time:0.0460.03
Connection time:0.0750.078
Pretransfer time:0.2210.18
Start transfer time:0.2530.273


United Kingdom, EUGoogle Cloud StorageAmazon S3
Total time:1.4625.137
DNS lookup time:0.0145.012
Connection time:1.0155.025
Pretransfer time:1.0275.061
Start transfer time:1.4425.108


Los Angeles, USGoogle Cloud StorageAmazon S3
Total time:0.3781.336
DNS lookup time:0.0320.193
Connection time:0.0330.349
Pretransfer time:0.0410.667
Start transfer time:0.2990.869


Kansas, USGoogle Cloud StorageAmazon S3
Total time:0.1450.815
DNS lookup time:0.0010.018
Connection time:0.0130.124
Pretransfer time:0.040.342
Start transfer time:0.1110.494


North Carolina, USGoogle Cloud StorageAmazon S3
Total time:0.10.95
DNS lookup time:0.0210.01
Connection time:0.030.135
Pretransfer time:0.0540.392
Start transfer time:0.0640.57


AFRICAGoogle Cloud StorageAmazon S3
Total time:8.0596.343
DNS lookup time:5.1795.178
Connection time:5.5925.339
Pretransfer time:6.4255.666
Start transfer time:6.8345.858


BRAZILGoogle Cloud StorageAmazon S3
Total time:0.2371.714
DNS lookup time:0.1410.237
Connection time:0.1530.441
Pretransfer time:0.1850.855
Start transfer time:0.1981.098


AUSTRALIAGoogle Cloud StorageAmazon S3
Total time:0.1742.952
DNS lookup time:0.1460.125
Connection time:0.1480.472
Pretransfer time:0.161.172
Start transfer time:0.1661.565


Taiwan, ASIAGoogle Cloud StorageAmazon S3
Total time:0.5475.053
DNS lookup time:0.0360.008
Connection time:0.0420.296
Pretransfer time:0.0630.886
Start transfer time:0.4371.198


Malaysia, ASIAGoogle Cloud StorageAmazon S3
Total time:1.2862.102
DNS lookup time:0.0060.005
Connection time:0.0090.289
Pretransfer time:0.0310.867
Start transfer time:1.0031.197


India, ASIAGoogle Cloud StorageAmazon S3
Total time:0.3291.067
DNS lookup time:0.120.06
Connection time:0.1490.198
Pretransfer time:0.210.477
Start transfer time:0.2390.654


10 December 2013

MySQL 5.6 - Slave: received end packet from server, apparent master shutdown

I received the following error on a replication setup with Percona MySQL 5.6:
[Note] Slave: received end packet from server, apparent master shutdown:
[Note] Slave I/O thread: Failed reading log event, reconnecting to retry, log 'mysql-bin.000024' at position 37418593

The issue was the server_uuid which is read from data_dir/auto.cnf. Read more here.
To diagnose:
mysql> show global variables like "Server%"
+----------------+--------------------------------------+
| Variable_name | Value |
+----------------+--------------------------------------+
| server_id | 20 |
| server_id_bits | 32 |
| server_uuid | 6a16558f-3b42-11e3-8548-00163e3d05d3 |
+----------------+--------------------------------------+
You have to make sure that server_id and server_uuid are different between all nodes participating in the master/slave setup.