MySQL auf einem vServer

[mc1]

Hallo alle zusammen :-D

Ich arbeite mich seit einiger Zeit in die Grundlagen zum Betrieb eines Servers ein und betreibe im Moment einen (nicht öffentlichen) vServer mit Apache2, PHP und MySQL.

Leider hat sich ein Problen aufgetan. Die Verbindung zu MySQL ist auf diesem vServer langsamer als auf einem handelsüblichen Webspace-Paket welches ich ebenfalls administriere. In einem Versuchsaufbau habe ich auf beiden jeweils 50.000 Inserts ausgeführt, Ergebnis: auf dem normalen Webspace benötigt MySQL lediglich 2.05 Sekunden, auf meinem vServer hingegen teilweise über 5 Sekunden.

Warum ich von einem Fehler von MySQL ausgehe? Bei normalen PHP-Scripten ist der vServer bei allen Tests schneller. Ich habe euch mal die Variablen aufgelistet, vllt. könnt ihr ja was erkennen.

Code: Select all

auto_increment_increment	1
auto_increment_offset	1
autocommit	ON
automatic_sp_privileges	ON
back_log	50
basedir	/usr
big_tables	OFF
binlog_cache_size	32768
binlog_direct_non_transactional_updates	OFF
binlog_format	STATEMENT
binlog_stmt_cache_size	32768
bulk_insert_buffer_size	8388608
character_set_client	utf8
character_set_connection	utf8
character_set_database	latin1
character_set_filesystem	binary
character_set_results	utf8
character_set_server	latin1
character_set_system	utf8
character_sets_dir	/usr/share/mysql/charsets/
collation_connection	utf8_general_ci
collation_database	latin1_swedish_ci
collation_server	latin1_swedish_ci
completion_type	NO_CHAIN
concurrent_insert	AUTO
connect_timeout	10
datadir	/var/lib/mysql/
date_format	%Y-%m-%d
datetime_format	%Y-%m-%d %H:%i:%s
default_storage_engine	InnoDB
default_week_format	0
delay_key_write	ON
delayed_insert_limit	100
delayed_insert_timeout	300
delayed_queue_size	1000
div_precision_increment	4
engine_condition_pushdown	ON
error_count	0
event_scheduler	OFF
expire_logs_days	10
external_user	
flush	OFF
flush_time	0
foreign_key_checks	ON
ft_boolean_syntax	+ -><()~*:""&|
ft_max_word_len	84
ft_min_word_len	4
ft_query_expansion_limit	20
ft_stopword_file	(built-in)
general_log	OFF
general_log_file	/var/lib/mysql/localhost.log
group_concat_max_len	1024
have_compress	YES
have_crypt	YES
have_csv	YES
have_dynamic_loading	YES
have_geometry	YES
have_innodb	YES
have_ndbcluster	NO
have_openssl	DISABLED
have_partitioning	YES
have_profiling	YES
have_query_cache	YES
have_rtree_keys	YES
have_ssl	DISABLED
have_symlink	YES
hostname	localhost.localdomain
identity	0
ignore_builtin_innodb	OFF
init_connect	
init_file	
init_slave	
innodb_adaptive_flushing	ON
innodb_adaptive_hash_index	ON
innodb_additional_mem_pool_size	8388608
innodb_autoextend_increment	8
innodb_autoinc_lock_mode	1
innodb_buffer_pool_instances	1
innodb_buffer_pool_size	268435456
innodb_change_buffering	all
innodb_checksums	ON
innodb_commit_concurrency	0
innodb_concurrency_tickets	500
innodb_data_file_path	ibdata1:10M:autoextend
innodb_data_home_dir	
innodb_doublewrite	ON
innodb_fast_shutdown	1
innodb_file_format	Antelope
innodb_file_format_check	ON
innodb_file_format_max	Antelope
innodb_file_per_table	OFF
innodb_flush_log_at_trx_commit	1
innodb_flush_method	
innodb_force_load_corrupted	OFF
innodb_force_recovery	0
innodb_io_capacity	200
innodb_large_prefix	OFF
innodb_lock_wait_timeout	50
innodb_locks_unsafe_for_binlog	OFF
innodb_log_buffer_size	4194304
 Variable_name 	 Value 
innodb_log_file_size	5242880
innodb_log_files_in_group	2
innodb_log_group_home_dir	./
innodb_max_dirty_pages_pct	75
innodb_max_purge_lag	0
innodb_mirrored_log_groups	1
innodb_old_blocks_pct	37
innodb_old_blocks_time	0
innodb_open_files	300
innodb_purge_batch_size	20
innodb_purge_threads	0
innodb_random_read_ahead	OFF
innodb_read_ahead_threshold	56
innodb_read_io_threads	4
innodb_replication_delay	0
innodb_rollback_on_timeout	OFF
innodb_rollback_segments	128
innodb_spin_wait_delay	6
innodb_stats_method	nulls_equal
innodb_stats_on_metadata	ON
innodb_stats_sample_pages	8
innodb_strict_mode	OFF
innodb_support_xa	ON
innodb_sync_spin_loops	30
innodb_table_locks	ON
innodb_thread_concurrency	0
innodb_thread_sleep_delay	10000
innodb_use_native_aio	OFF
innodb_use_sys_malloc	ON
innodb_version	1.1.8
innodb_write_io_threads	4
insert_id	0
interactive_timeout	28800
join_buffer_size	131072
keep_files_on_create	OFF
key_buffer_size	16777216
key_cache_age_threshold	300
key_cache_block_size	1024
key_cache_division_limit	100
large_files_support	ON
large_page_size	0
large_pages	OFF
last_insert_id	0
lc_messages	en_US
lc_messages_dir	/usr/share/mysql/
lc_time_names	en_US
license	GPL
local_infile	ON
lock_wait_timeout	31536000
locked_in_memory	OFF
log	OFF
log_bin	OFF
log_bin_trust_function_creators	OFF
log_error	
log_output	FILE
log_queries_not_using_indexes	OFF
log_slave_updates	OFF
log_slow_queries	OFF
log_warnings	1
long_query_time	10.000000
low_priority_updates	OFF
lower_case_file_system	OFF
lower_case_table_names	0
max_allowed_packet	16777216
max_binlog_cache_size	18446744073709547520
max_binlog_size	104857600
max_binlog_stmt_cache_size	18446744073709547520
max_connect_errors	10
max_connections	151
max_delayed_threads	20
max_error_count	64
max_heap_table_size	16777216
max_insert_delayed_threads	20
max_join_size	18446744073709551615
max_length_for_sort_data	1024
max_long_data_size	16777216
max_prepared_stmt_count	16382
max_relay_log_size	0
max_seeks_for_key	18446744073709551615
max_sort_length	1024
max_sp_recursion_depth	0
max_tmp_tables	32
max_user_connections	0
max_write_lock_count	18446744073709551615
metadata_locks_cache_size	1024
min_examined_row_limit	0
multi_range_count	256
myisam_data_pointer_size	6
myisam_max_sort_file_size	9223372036853727232
myisam_mmap_size	18446744073709551615
myisam_recover_options	BACKUP
myisam_repair_threads	1
myisam_sort_buffer_size	8388608
myisam_stats_method	nulls_unequal
myisam_use_mmap	OFF
net_buffer_length	16384
net_read_timeout	30
net_retry_count	10
net_write_timeout	60
new	OFF
 Variable_name 	 Value 
old	OFF
old_alter_table	OFF
old_passwords	OFF
open_files_limit	1024
optimizer_prune_level	1
optimizer_search_depth	62
optimizer_switch	index_merge=on,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,engine_condition_pushdown=on
performance_schema	OFF
performance_schema_events_waits_history_long_size	10000
performance_schema_events_waits_history_size	10
performance_schema_max_cond_classes	80
performance_schema_max_cond_instances	1000
performance_schema_max_file_classes	50
performance_schema_max_file_handles	32768
performance_schema_max_file_instances	10000
performance_schema_max_mutex_classes	200
performance_schema_max_mutex_instances	1000000
performance_schema_max_rwlock_classes	30
performance_schema_max_rwlock_instances	1000000
performance_schema_max_table_handles	100000
performance_schema_max_table_instances	50000
performance_schema_max_thread_classes	50
performance_schema_max_thread_instances	1000
pid_file	/var/run/mysqld/mysqld.pid
plugin_dir	/usr/lib/mysql/plugin/
port	3306
preload_buffer_size	32768
profiling	OFF
profiling_history_size	15
protocol_version	10
proxy_user	
pseudo_thread_id	140
query_alloc_block_size	8192
query_cache_limit	1048576
query_cache_min_res_unit	4096
query_cache_size	16777216
query_cache_type	ON
query_cache_wlock_invalidate	OFF
query_prealloc_size	8192
rand_seed1	0
rand_seed2	0
range_alloc_block_size	4096
read_buffer_size	131072
read_only	OFF
read_rnd_buffer_size	262144
relay_log	
relay_log_index	
relay_log_info_file	relay-log.info
relay_log_purge	ON
relay_log_recovery	OFF
relay_log_space_limit	0
report_host	
report_password	
report_port	3306
report_user	
rpl_recovery_rank	0
secure_auth	OFF
secure_file_priv	
server_id	0
skip_external_locking	ON
skip_name_resolve	OFF
skip_networking	OFF
skip_show_database	OFF
slave_compressed_protocol	OFF
slave_exec_mode	STRICT
slave_load_tmpdir	/tmp
slave_net_timeout	3600
slave_skip_errors	OFF
slave_transaction_retries	10
slave_type_conversions	
slow_launch_time	2
slow_query_log	OFF
slow_query_log_file	/var/lib/mysql/localhost-slow.log
socket	/var/run/mysqld/mysqld.sock
sort_buffer_size	2097152
sql_auto_is_null	OFF
sql_big_selects	ON
sql_big_tables	OFF
sql_buffer_result	OFF
sql_log_bin	ON
sql_log_off	OFF
sql_low_priority_updates	OFF
sql_max_join_size	18446744073709551615
sql_mode	
sql_notes	ON
sql_quote_show_create	ON
sql_safe_updates	OFF
sql_select_limit	18446744073709551615
sql_slave_skip_counter	0
sql_warnings	OFF
ssl_ca	
ssl_capath	
ssl_cert	
ssl_cipher	
ssl_key	
storage_engine	InnoDB
stored_program_cache	256
sync_binlog	0
sync_frm	ON
sync_master_info	0
 Variable_name 	 Value 
sync_relay_log	0
sync_relay_log_info	0
system_time_zone	UTC
table_definition_cache	400
table_open_cache	400
thread_cache_size	8
thread_concurrency	10
thread_handling	one-thread-per-connection
thread_stack	196608
time_format	%H:%i:%s
time_zone	SYSTEM
timed_mutexes	OFF
timestamp	1343756282
tmp_table_size	16777216
tmpdir	/tmp
transaction_alloc_block_size	8192
transaction_prealloc_size	4096
tx_isolation	REPEATABLE-READ
unique_checks	ON
updatable_views_with_limit	YES
version	5.5.24-0ubuntu0.12.04.1
version_comment	(Ubuntu)
version_compile_machine	x86_64
version_compile_os	debian-linux-gnu
wait_timeout	28800
warning_count	0

Betriebssystem ist Ubuntu 12, Serverstandort ist Düsseldorf.

Was sagt ihr dazu? :)

[rudelgurke]

Mal tuning-primer und mysqltuner befragt ? Und wie sieht die IO Last aus ?

[mc1]

Ja, außer eine Meldung nichts besonderes:

Code: Select all

[!!] Total fragmented tables: 1

[Joe User]

Bitte die vollständigen Ausgaben beider Scripts und die my.cnf posten, danke.

[mc1]

Tuning-Primer:

SLOW QUERIES
The slow query log is NOT enabled.
Current long_query_time = 10.000000 sec.
You have 0 out of 350332 that take longer than 10.000000 sec. to complete
Your long_query_time seems to be fine

BINARY UPDATE LOG
The binary update log is NOT enabled.
You will not be able to do point in time recovery
See http://dev.mysql.com/doc/refman/5.5/en/ ... overy.html

WORKER THREADS
Current thread_cache_size = 8
Current threads_cached = 3
Current threads_per_sec = 0
Historic threads_per_sec = 0
Your thread_cache_size is fine

MAX CONNECTIONS
Current max_connections = 151
Current threads_connected = 1
Historic max_used_connections = 4
The number of used connections is 2% of the configured maximum.
You are using less than 10% of your configured max_connections.
Lowering max_connections could help to avoid an over-allocation of memory
See "MEMORY USAGE" section to make sure you are not over-allocating

INNODB STATUS
Current InnoDB index space = 0 bytes
Current InnoDB data space = 1 M
Current InnoDB buffer pool free = 94 %
Current innodb_buffer_pool_size = 256 M
Depending on how much space your innodb indexes take up it may be safe
to increase this value to up to 2 / 3 of total system memory

MEMORY USAGE
Max Memory Ever Allocated : 310 M
Configured Max Per-thread Buffers : 405 M
Configured Max Global Buffers : 300 M
Configured Max Memory Limit : 705 M
Physical Memory : 487 M

Max memory limit exceeds 90% of physical memory

KEY BUFFER
No key reads?!
Seriously look into using some indexes
Current MyISAM index space = 132 K
Current key_buffer_size = 16 M
Key cache miss rate is 1 : 0
Key buffer free ratio = 81 %
Your key_buffer_size seems to be fine

QUERY CACHE
Query cache is enabled
Current query_cache_size = 16 M
Current query_cache_used = 29 K
Current query_cache_limit = 1 M
Current Query cache Memory fill ratio = .18 %
Current query_cache_min_res_unit = 4 K
Your query_cache_size seems to be too high.
Perhaps you can use these resources elsewhere
MySQL won't cache query results that are larger than query_cache_limit in size

SORT OPERATIONS
Current sort_buffer_size = 2 M
Current read_rnd_buffer_size = 256 K
No sort operations have been performed
Sort buffer seems to be fine

JOINS
Current join_buffer_size = 132.00 K
You have had 0 queries where a join could not use an index properly
Your joins seem to be using indexes properly

OPEN FILES LIMIT
Current open_files_limit = 1024 files
The open_files_limit should typically be set to at least 2x-3x
that of table_cache if you have heavy MyISAM usage.
Your open_files_limit value seems to be fine

TABLE CACHE
Current table_open_cache = 400 tables
Current table_definition_cache = 400 tables
You have a total of 52 tables
You have 52 open tables.
The table_cache value seems to be fine

TEMP TABLES
Current max_heap_table_size = 16 M
Current tmp_table_size = 16 M
Of 480 temp tables, 17% were created on disk
Created disk tmp tables ratio seems fine

TABLE SCANS
Current read_buffer_size = 128 K
Current table scan ratio = 9 : 1
read_buffer_size seems to be fine

TABLE LOCKING
Current Lock Wait ratio = 0 : 350590
Your table locking seems to be fine

MySQL-Tuner:

[OK] Logged in using credentials from debian maintenance account.

-------- General Statistics --------------------------------------------------
[--] Skipped version check for MySQLTuner script
[OK] Currently running supported MySQL version 5.5.24-0ubuntu0.12.04.1
[OK] Operating on 64-bit architecture

-------- Storage Engine Statistics -------------------------------------------
[--] Status: +Archive -BDB -Federated +InnoDB -ISAM -NDBCluster
[--] Data in MyISAM tables: 28B (Tables: 10)
[--] Data in InnoDB tables: 1M (Tables: 1)
[--] Data in PERFORMANCE_SCHEMA tables: 0B (Tables: 17)
[!!] Total fragmented tables: 1

-------- Security Recommendations -------------------------------------------
[!!] User '@localhost' has no password set.
[!!] User '@localhost.localdomain' has no password set.

-------- Performance Metrics -------------------------------------------------
[--] Up for: 213503982334601d 0h 58m 50s (350K q [0.000 qps], 197 conn, TX: 4M, RX: 19M)
[--] Reads / Writes: 0% / 100%
[--] Total buffers: 316.0M global + 2.7M per thread (151 max threads)
[!!] Maximum possible memory usage: 721.8M (148% of installed RAM)
[OK] Slow queries: 0% (0/350K)
[OK] Highest usage of available connections: 2% (4/151)
[OK] Key buffer size / total MyISAM indexes: 16.0M/123.0K
[!!] Query cache efficiency: 1.6% (4 cached / 246 selects)
[OK] Query cache prunes per day: 0
[OK] Temporary tables created on disk: 17% (104 on disk / 596 total)
[OK] Thread cache hit rate: 97% (4 created / 197 connections)
[!!] Table cache hit rate: 19% (52 open / 269 opened)
[OK] Open file limit used: 6% (68/1K)
[OK] Table locks acquired immediately: 100% (93 immediate / 93 locks)
[OK] InnoDB data size / buffer pool: 1.5M/256.0M

-------- Recommendations -----------------------------------------------------
General recommendations:
Run OPTIMIZE TABLE to defragment tables for better performance
Reduce your overall MySQL memory footprint for system stability
Enable the slow query log to troubleshoot bad queries
Increase table_cache gradually to avoid file descriptor limits
Variables to adjust:
*** MySQL's maximum memory usage is dangerously high ***
*** Add RAM before increasing MySQL buffer variables ***
query_cache_limit (> 1M, or use smaller result sets)
table_cache (> 400)

my.cnf

#
# The MySQL database server configuration file.
#
# You can copy this to one of:
# - "/etc/mysql/my.cnf" to set global options,
# - "~/.my.cnf" to set user-specific options.
#
# One can use all long options that the program supports.
# Run program with --help to get a list of available options and with
# --print-defaults to see which it would actually understand and use.
#
# For explanations see
# http://dev.mysql.com/doc/mysql/en/serve ... ables.html

# This will be passed to all mysql clients
# It has been reported that passwords should be enclosed with ticks/quotes
# escpecially if they contain "#" chars...
# Remember to edit /etc/mysql/debian.cnf when changing the socket location.
[client]
port = 3306
socket = /var/run/mysqld/mysqld.sock

# Here is entries for some specific programs
# The following values assume you have at least 32M ram

# This was formally known as [safe_mysqld]. Both versions are currently parsed.
[mysqld_safe]
socket = /var/run/mysqld/mysqld.sock
nice = 0

[mysqld]
#
# * Basic Settings
#
user = mysql
pid-file = /var/run/mysqld/mysqld.pid
socket = /var/run/mysqld/mysqld.sock
port = 3306
basedir = /usr
datadir = /var/lib/mysql
tmpdir = /tmp
lc-messages-dir = /usr/share/mysql
skip-external-locking
innodb_log_file_size=5M
innodb_buffer_pool_size=256M
innodb_log_buffer_size=4M
#
# Instead of skip-networking the default is now to listen only on
# localhost which is more compatible and is not less secure.
bind-address = 127.0.0.1
#
# * Fine Tuning
#
key_buffer = 16M
max_allowed_packet = 16M
thread_stack = 192K
thread_cache_size = 8
# This replaces the startup script and checks MyISAM tables if needed
# the first time they are touched
myisam-recover = BACKUP
#max_connections = 100
#table_cache = 64
#thread_concurrency = 10
#
# * Query Cache Configuration
#
query_cache_limit = 1M
query_cache_size = 16M
#
# * Logging and Replication
#
# Both location gets rotated by the cronjob.
# Be aware that this log type is a performance killer.
# As of 5.1 you can enable the log at runtime!
#general_log_file = /var/log/mysql/mysql.log
#general_log = 1
#
# Error logging goes to syslog due to /etc/mysql/conf.d/mysqld_safe_syslog.cnf.
#
# Here you can see queries with especially long duration
#log_slow_queries = /var/log/mysql/mysql-slow.log
#long_query_time = 2
#log-queries-not-using-indexes
#
# The following can be used as easy to replay backup logs or for replication.
# note: if you are setting up a replication slave, see README.Debian about
# other settings you may need to change.
#server-id = 1
#log_bin = /var/log/mysql/mysql-bin.log
expire_logs_days = 10
max_binlog_size = 100M
#binlog_do_db = include_database_name
#binlog_ignore_db = include_database_name
#
# * InnoDB
#
# InnoDB is enabled by default with a 10MB datafile in /var/lib/mysql/.
# Read the manual for more InnoDB related options. There are many!
#
# * Security Features
#
# Read the manual, too, if you want chroot!
# chroot = /var/lib/mysql/
#
# For generating SSL certificates I recommend the OpenSSL GUI "tinyca".
#
# ssl-ca=/etc/mysql/cacert.pem
# ssl-cert=/etc/mysql/server-cert.pem
# ssl-key=/etc/mysql/server-key.pem



[mysqldump]
quick
quote-names
max_allowed_packet = 16M

[mysql]
#no-auto-rehash # faster start of mysql but no tab completition

[isamchk]
key_buffer = 16M

#
# * IMPORTANT: Additional settings that can override those from this file!
# The files must end with '.cnf', otherwise they'll be ignored.
#
!includedir /etc/mysql/conf.d/

[Joe User]

Bitte die maximalen Connections senken:
max_connections=20

Den Query-Cache kannst Du vermutlich komplett deaktivieren, ausprobieren bitte.

Mehr ist nicht sinnvoll.

[ddm3ve]

Code: Select all

MEMORY USAGE
Max Memory Ever Allocated : 310 M
Configured Max Per-thread Buffers : 405 M
Configured Max Global Buffers : 300 M
Configured Max Memory Limit : 705 M
Physical Memory : 487 M

Max memory limit exceeds 90% of physical memory

Wenn es dumm läuft, crasht Dir Deine Datenbank mit Datacorruption.

Wieviele RAM hat das System denn garantiert?

[Joe User]

Wenn er die max_connections auf 20 limitiert, kommt er auf maximal 370MB RAM für MySQL (laut mysqltuner). Das sollte das Risiko eines Kills stark verringern.

[ddm3ve]

Kommt immer auf den Anwendungsfall an.
Bei 487 phyischem RAM, eventuell aber nur 250MB garantiertem RAM kann das durchaus zum Problem werden.

Aber selbst 130MB sind fürs Os schon knapp. Plant man den Betrieb einer Webapplikation, da reicht auch schon PHP mit Fastcgi oder dergleichen, dann kann der Rahmen schnell ausgereizt werden.

Und ich hbe nicht den eindruck, dass er hier nur einen dedizierten Datenbankserver betreibt.
Seine Lags können aus unterschiedlichsten gründen entstehen.
Ein vServer ist halt nunmal ein vServer mit all seinen Macken.
Einen "billigen" Storage auf dem mehrere User rum rutschen z.B.

Eine gute Optimierung eines solchen Systemes kann durchaus einige Zeit in anspruch nehmen, da fängt es bei Filestem an, geht über die Konfiguration bis hin zu richtiger Indizierung denormalisierung / Datenbankdesign.

Warum der "schnöde" Webspace so toll und schnell funktioniert ist u.U. der Grund, dass dahinter ein anderes Blech (auch der Storage) und ein dedizierter DB Server steht, viel RAM für den Query Cache und jede Kiste antwortet u.U. flott aus dem Cache. RAM ist nunmal schneller als Platte.

Die Schwierkeit aus einem vServr eine Rennsemmel zu machen ist nunmal die, dass man häufig sehr hohe dynamische Versprechen bekommen hat, aber zuverlässig oft nur ein "Bruchteil" zugesichert wird.

Damit hat man nur die Möglichkeit auf das garantiert verfügbare hin zu optimieren.
Der Spielraum ist dann meist sehr klein. Darüber muss man sich im klaren sein und vServer versus "guter Profi Hoster" Webspace, der vServer verliert an der Stelle oft oder kostet dann halt auch angemessen Geld.