Simple and easy way to put google analytics code on SMF 1. Download Global Headers and Footer Mod http://custom.simplemachines.org/mods/index.php?mod=351
2. Go to your Forum and login as a Administrator
3. Go to admin >> main >> pakages >> download pakages >> pakage to upload
4. upload the Global Headers and footer's zip file that you downloaded first.
5. Apply Mod to install.
6. If Mod is installed sucessfully then go to admin >> Configuration >> Global Headers and footer.
7. Paste your Google Analytics code in header or footer.
8. Check either Google Analytics code is installed correctly or not. to do so you have to login to your google analytics account and check the status.
I'm 100% sure that it works because of that mod.
Monday, April 27, 2009
Wednesday, April 22, 2009
Exim Console
Here are some useful things to know for managing an Exim 4 server. This assumes a prior working knowledge of SMTP, MTAs, and a UNIX shell prompt.Message-IDs and spool files
The message-IDs that Exim uses to refer to messages in its queue are mixed-case alpha-numeric, and take the form of: XXXXXX-YYYYYY-ZZ. Most commands related to managing the queue and logging use these message-ids.
There are three -- count 'em, THREE -- files for each message in the spool directory. If you're dealing with these files by hand, instead of using the appropriate exim commands as detailed below, make sure you get them all, and don't leave Exim with remnants of messages in the queue. I used to mess directly with these files when I first started running Exim machines, but thanks to the utilities described below, I haven't needed to do that in many months.
Files in /var/spool/exim/msglog contain logging information for each message and are named the same as the message-id.
Files in /var/spool/exim/input are named after the message-id, plus a suffix denoting whether it is the envelope header (-H) or message data (-D).
These directories may contain further hashed subdirectories to deal with larger mail queues, so don't expect everything to always appear directly in the top /var/spool/exim/input or /var/spool/exim/msglog directories; any searches or greps will need to be recursive. See if there is a proper way to do what you're doing before working directly on the spool files.Basic information
Print a count of the messages in the queue:
root@localhost# exim -bpc
Print a listing of the messages in the queue (time queued, size, message-id, sender, recipient):
root@localhost# exim -bp
Print a summary of messages in the queue (count, volume, oldest, newest, domain, and totals):
root@localhost# exim -bp exiqsumm
Print what Exim is doing right now:
root@localhost# exiwhat
Test how exim will route a given address:
root@localhost# exim -bt alias@localdomain.comuser@thishost.com <-- alias@localdomain.com router = localuser, transport = local_deliveryroot@localhost# exim -bt user@thishost.comuser@thishost.com router = localuser, transport = local_deliveryroot@localhost# exim -bt user@remotehost.com router = lookuphost, transport = remote_smtp host mail.remotehost.com [1.2.3.4] MX=0
Run a pretend SMTP transaction from the command line, as if it were coming from the given IP address. This will display Exim's checks, ACLs, and filters as they are applied. The message will NOT actually be delivered.
root@localhost# exim -bh 192.168.11.22
Display all of Exim's configuration settings:
root@localhost# exim -bP
Searching the queue with exiqgrep
Exim includes a utility that is quite nice for grepping through the queue, called exiqgrep. Learn it. Know it. Live it. If you're not using this, and if you're not familiar with the various flags it uses, you're probably doing things the hard way, like piping `exim -bp` into awk, grep, cut, or `wc -l`. Don't make life harder than it already is.
First, various flags that control what messages are matched. These can be combined to come up with a very particular search.
Use -f to search the queue for messages from a specific sender:
root@localhost# exiqgrep -f [luser]@domain
Use -r to search the queue for messages for a specific recipient/domain:
root@localhost# exiqgrep -r [luser]@domain
Use -o to print messages older than the specified number of seconds. For example, messages older than 1 day:
root@localhost# exiqgrep -o 86400 [...]
Use -y to print messages that are younger than the specified number of seconds. For example, messages less than an hour old:
root@localhost# exiqgrep -y 3600 [...]
Use -s to match the size of a message with a regex. For example, 700-799 bytes:
root@localhost# exiqgrep -s '^7..$' [...]
Use -z to match only frozen messages, or -x to match only unfrozen messages.
There are also a few flags that control the display of the output.
Use -i to print just the message-id as a result of one of the above two searches:
root@localhost# exiqgrep -i [ -r -f ] ...
Use -c to print a count of messages matching one of the above searches:
root@localhost# exiqgrep -c ...
Print just the message-id of the entire queue:
root@localhost# exiqgrep -i
Managing the queue
The main exim binary (/usr/sbin/exim) is used with various flags to make things happen to messages in the queue. Most of these require one or more message-IDs to be specified in the command line, which is where `exiqgrep -i` as described above really comes in handy.
Start a queue run:
root@localhost# exim -q -v
Start a queue run for just local deliveries:
root@localhost# exim -ql -v
Remove a message from the queue:
root@localhost# exim -Mrm [ ... ]
Freeze a message:
root@localhost# exim -Mf [ ... ]
Thaw a message:
root@localhost# exim -Mt [ ... ]
Deliver a message, whether it's frozen or not, whether the retry time has been reached or not:
root@localhost# exim -M [ ... ]
Deliver a message, but only if the retry time has been reached:
root@localhost# exim -Mc [ ... ]
Force a message to fail and bounce as "cancelled by administrator":
root@localhost# exim -Mg [ ... ]
Remove all frozen messages:
root@localhost# exiqgrep -z -i xargs exim -Mrm
Remove all messages older than five days (86400 * 5 = 432000 seconds):
root@localhost# exiqgrep -o 432000 -i xargs exim -Mrm
Freeze all queued mail from a given sender:
root@localhost# exiqgrep -i -f luser@example.tld xargs exim -Mf
View a message's headers:
root@localhost# exim -Mvh
View a message's body:
root@localhost# exim -Mvb
View a message's logs:
root@localhost# exim -Mvl
Add a recipient to a message:
root@localhost# exim -Mar [ ... ]
Edit the sender of a message:
root@localhost# exim -Mes
Access control
Exim allows you to apply access control lists at various points of the SMTP transaction by specifying an ACL to use and defining its conditions in exim.conf. You could start with the HELO string.
# Specify the ACL to use after HELOacl_smtp_helo = check_helo
# Conditions for the check_helo ACL:check_helo:
deny message = Gave HELO/EHLO as "friend" log_message = HELO/EHLO friend condition = ${if eq {$sender_helo_name}{friend} {yes}{no}}
deny message = Gave HELO/EHLO as our IP address log_message = HELO/EHLO our IP address condition = ${if eq {$sender_helo_name}{$interface_address} {yes}{no}}
accept
NOTE: Pursue HELO checking at your own peril. The HELO is fairly unimportant in the grand scheme of SMTP these days, so don't put too much faith in whatever it contains. Some spam might seem to use a telltale HELO string, but you might be surprised at how many legitimate messages start off with a questionable HELO as well. Anyway, it's just as easy for a spammer to send a proper HELO than it is to send HELO im.a.spammer, so consider yourself lucky if you're able to stop much spam this way.
Next, you can perform a check on the sender address or remote host. This shows how to do that after the RCPT TO command; if you reject here, as opposed to rejecting after the MAIL FROM, you'll have better data to log, such as who the message was intended for.
# Specify the ACL to use after RCPT TOacl_smtp_rcpt = check_recipient
# Conditions for the check_recipient ACLcheck_recipient:
# [...]
drop hosts = /etc/exim_reject_hosts drop senders = /etc/exim_reject_senders
# [ Probably a whole lot more... ]
This example uses two plain text files as blacklists. Add appropriate entries to these files - hostnames/IP addresses to /etc/exim_reject_hosts, addresses to /etc/exim_reject_senders, one entry per line.
It is also possible to perform content scanning using a regex against the body of a message, though obviously this can cause Exim to use more CPU than it otherwise would need to, especially on large messages.
# Specify the ACL to use after DATAacl_smtp_data = check_message
# Conditions for the check_messages ACLcheck_message:
deny message = "Sorry, Charlie: $regex_match_string" regex = ^Subject:: .*Lower your self-esteem by becoming a sysadmin
accept
Fix SMTP-Auth for Pine
If pine can't use SMTP authentication on an Exim host and just returns an "unable to authenticate" message without even asking for a password, add the following line to exim.conf:
begin authenticators
fixed_plain: driver = plaintext public_name = PLAIN server_condition = "${perl{checkuserpass}{$1}{$2}{$3}}" server_set_id = $2> server_prompts = :
This was a problem on CPanel Exim builds awhile ago, but they seem to have added this line to their current stock configuration.Log the subject line
This is one of the most useful configuration tweaks I've ever found for Exim. Add this to exim.conf, and you can log the subject lines of messages that pass through your server. This is great for troubleshooting, and for getting a very rough idea of what messages may be spam.
log_selector = +subject
Reducing or increasing what is logged.Disable identd lookups
Frankly, I don't think identd has been useful for a long time, if ever. Identd relies on the connecting host to confirm the identity (system UID) of the remote user who owns the process that is making the network connection. This may be of some use in the world of shell accounts and IRC users, but it really has no place on a high-volume SMTP server, where the UID is often simply "mail" or whatever the remote MTA runs as, which is useless to know. It's overhead, and results in nothing but delays while the identd query is refused or times out. You can stop your Exim server from making these queries by setting the timeout to zero seconds in exim.conf:
rfc1413_query_timeout = 0s
Disable Attachment Blocking
To disable the executable-attachment blocking that many Cpanel servers do by default but don't provide any controls for on a per-domain basis, add the following block to the beginning of the /etc/antivirus.exim file:
if $header_to: matches "example\.comexample2\.com"then finishendif
It is probably possible to use a separate file to list these domains, but I haven't had to do this enough times to warrant setting such a thing up.Searching the logs with exigrep
The exigrep utility (not to be confused with exiqgrep) is used to search an exim log for a string or pattern. It will print all log entries with the same internal message-id as those that matched the pattern, which is very handy since any message will take up at least three lines in the log. exigrep will search the entire content of a log entry, not just particular fields.
One can search for messages sent from a particular IP address:
root@localhost# exigrep '<= .* \[12.34.56.78\] ' /path/to/exim_log
Search for messages sent to a particular IP address:
root@localhost# exigrep '=> .* \[12.34.56.78\]' /path/to/exim_log
This example searches for outgoing messages, which have the "=>" symbol, sent to "user@domain.tld". The pipe to grep for the "<=" symbol will match only the lines with information on the sender - the From address, the sender's IP address, the message size, the message ID, and the subject line if you have enabled logging the subject. The purpose of doing such a search is that the desired information is not on the same log line as the string being searched for.
root@localhost# exigrep '=> .*user@domain.tld' /path/to/exim_log fgrep '<='
Generate and display Exim stats from a logfile:
root@localhost# eximstats /path/to/exim_mainlog
Same as above, with less verbose output:
root@localhost# eximstats -ne -nr -nt /path/to/exim_mainlog
Same as above, for one particular day:
root@localhost# fgrep YYYY-MM-DD /path/to/exim_mainlog eximstats
Bonus!
To delete all queued messages containing a certain string in the body:
root@localhost# grep -lr 'a certain string' /var/spool/exim/input/ \ sed -e 's/^.*\/\([a-zA-Z0-9-]*\)-[DH]$/\1/g' xargs exim -Mrm
Note that the above only delves into /var/spool/exim in order to grep for queue files with the given string, and that's just because exiqgrep doesn't have a feature to grep the actual bodies of messages. If you are deleting these files directly, YOU ARE DOING IT WRONG! Use the appropriate exim command to properly deal with the queue.
If you have to feed many, many message-ids (such as the output of an `exiqgrep -i` command that returns a lot of matches) to an exim command, you may exhaust the limit of your shell's command line arguments. In that case, pipe the listing of message-ids into xargs to run only a limited number of them at once. For example, to remove thousands of messages sent from joe@example.com:
root@localhost# exiqgrep -i -f '<'">joe@example.com>' xargs exim -Mrm
Speaking of "DOING IT WRONG" -- Attention, CPanel forum readers
I get a number of hits to this page from a link in this post at the CPanel forums. The question is:
Due to spamming, spoofing from fields, etc., etc., etc., I am finding it necessary to spend more time to clear the exim queue from time to time. [...] what command would I use to delete the queue
The answer is: Just turn exim off, because your customers are better off knowing that email simply isn't running on your server, than having their queued messages deleted without notice.
Or, figure out what is happening. The examples given in that post pay no regard to the legitimacy of any message, they simply delete everything, making the presumption that if a message is in the queue, it's junk. That is total fallacy. There are a number of reasons legitimate mail can end up in the queue. Maybe your backups or CPanel's "upcp" process are running, and your load average is high -- exim goes into a queue-only mode at a certain threshold, where it stops trying to deliver messages as they come in and just queues them until the load goes back down. Or, maybe it's an outgoing message, and the DNS lookup failed, or the connection to the domain's MX failed, or maybe the remote MX is busy or greylisting you with a 4xx deferral. These are all temporary failures, not permanent ones, and the whole point of having temporary failures in SMTP and a mail queue in your MTA is to be able to try again after awhile.
Exim already purges messages from the queue after the period of time specified in exim.conf. If you have this value set appropriately, there is absolutely no point in removing everything from your queue every day with a cron job. You will lose legitimate mail, and the sender and recipient will never know if or why it happened. Do not do this!
If you regularly have a large number of messages in your queue, find out why they are there. If they are outbound messages, see who is sending them, where they're addressed to, and why they aren't getting there. If they are inbound messages, find out why they aren't getting delivered to your user's account. If you need to delete some, use exiqgrep to pick out just the ones that should be deleted.Reload the configuration
After making changes to exim.conf, you need to give the main exim pid a SIGHUP to re-exec it and have the configuration re-read. Sure, you could stop and start the service, but that's overkill and causes a few seconds of unnecessary downtime. Just do this:
root@localhost# kill -HUP `cat /var/spool/exim/exim-daemon.pid`
You should then see something resembling the following in exim_mainlog:
pid 1079: SIGHUP received: re-exec daemonexim 4.52 daemon started: pid=1079, -q1h, listening for SMTP on port 25 (IPv4)
The message-IDs that Exim uses to refer to messages in its queue are mixed-case alpha-numeric, and take the form of: XXXXXX-YYYYYY-ZZ. Most commands related to managing the queue and logging use these message-ids.
There are three -- count 'em, THREE -- files for each message in the spool directory. If you're dealing with these files by hand, instead of using the appropriate exim commands as detailed below, make sure you get them all, and don't leave Exim with remnants of messages in the queue. I used to mess directly with these files when I first started running Exim machines, but thanks to the utilities described below, I haven't needed to do that in many months.
Files in /var/spool/exim/msglog contain logging information for each message and are named the same as the message-id.
Files in /var/spool/exim/input are named after the message-id, plus a suffix denoting whether it is the envelope header (-H) or message data (-D).
These directories may contain further hashed subdirectories to deal with larger mail queues, so don't expect everything to always appear directly in the top /var/spool/exim/input or /var/spool/exim/msglog directories; any searches or greps will need to be recursive. See if there is a proper way to do what you're doing before working directly on the spool files.Basic information
Print a count of the messages in the queue:
root@localhost# exim -bpc
Print a listing of the messages in the queue (time queued, size, message-id, sender, recipient):
root@localhost# exim -bp
Print a summary of messages in the queue (count, volume, oldest, newest, domain, and totals):
root@localhost# exim -bp exiqsumm
Print what Exim is doing right now:
root@localhost# exiwhat
Test how exim will route a given address:
root@localhost# exim -bt alias@localdomain.comuser@thishost.com <-- alias@localdomain.com router = localuser, transport = local_deliveryroot@localhost# exim -bt user@thishost.comuser@thishost.com router = localuser, transport = local_deliveryroot@localhost# exim -bt user@remotehost.com router = lookuphost, transport = remote_smtp host mail.remotehost.com [1.2.3.4] MX=0
Run a pretend SMTP transaction from the command line, as if it were coming from the given IP address. This will display Exim's checks, ACLs, and filters as they are applied. The message will NOT actually be delivered.
root@localhost# exim -bh 192.168.11.22
Display all of Exim's configuration settings:
root@localhost# exim -bP
Searching the queue with exiqgrep
Exim includes a utility that is quite nice for grepping through the queue, called exiqgrep. Learn it. Know it. Live it. If you're not using this, and if you're not familiar with the various flags it uses, you're probably doing things the hard way, like piping `exim -bp` into awk, grep, cut, or `wc -l`. Don't make life harder than it already is.
First, various flags that control what messages are matched. These can be combined to come up with a very particular search.
Use -f to search the queue for messages from a specific sender:
root@localhost# exiqgrep -f [luser]@domain
Use -r to search the queue for messages for a specific recipient/domain:
root@localhost# exiqgrep -r [luser]@domain
Use -o to print messages older than the specified number of seconds. For example, messages older than 1 day:
root@localhost# exiqgrep -o 86400 [...]
Use -y to print messages that are younger than the specified number of seconds. For example, messages less than an hour old:
root@localhost# exiqgrep -y 3600 [...]
Use -s to match the size of a message with a regex. For example, 700-799 bytes:
root@localhost# exiqgrep -s '^7..$' [...]
Use -z to match only frozen messages, or -x to match only unfrozen messages.
There are also a few flags that control the display of the output.
Use -i to print just the message-id as a result of one of the above two searches:
root@localhost# exiqgrep -i [ -r -f ] ...
Use -c to print a count of messages matching one of the above searches:
root@localhost# exiqgrep -c ...
Print just the message-id of the entire queue:
root@localhost# exiqgrep -i
Managing the queue
The main exim binary (/usr/sbin/exim) is used with various flags to make things happen to messages in the queue. Most of these require one or more message-IDs to be specified in the command line, which is where `exiqgrep -i` as described above really comes in handy.
Start a queue run:
root@localhost# exim -q -v
Start a queue run for just local deliveries:
root@localhost# exim -ql -v
Remove a message from the queue:
root@localhost# exim -Mrm
Freeze a message:
root@localhost# exim -Mf
Thaw a message:
root@localhost# exim -Mt
Deliver a message, whether it's frozen or not, whether the retry time has been reached or not:
root@localhost# exim -M
Deliver a message, but only if the retry time has been reached:
root@localhost# exim -Mc
Force a message to fail and bounce as "cancelled by administrator":
root@localhost# exim -Mg
Remove all frozen messages:
root@localhost# exiqgrep -z -i xargs exim -Mrm
Remove all messages older than five days (86400 * 5 = 432000 seconds):
root@localhost# exiqgrep -o 432000 -i xargs exim -Mrm
Freeze all queued mail from a given sender:
root@localhost# exiqgrep -i -f luser@example.tld xargs exim -Mf
View a message's headers:
root@localhost# exim -Mvh
View a message's body:
root@localhost# exim -Mvb
View a message's logs:
root@localhost# exim -Mvl
Add a recipient to a message:
root@localhost# exim -Mar
Edit the sender of a message:
root@localhost# exim -Mes
Access control
Exim allows you to apply access control lists at various points of the SMTP transaction by specifying an ACL to use and defining its conditions in exim.conf. You could start with the HELO string.
# Specify the ACL to use after HELOacl_smtp_helo = check_helo
# Conditions for the check_helo ACL:check_helo:
deny message = Gave HELO/EHLO as "friend" log_message = HELO/EHLO friend condition = ${if eq {$sender_helo_name}{friend} {yes}{no}}
deny message = Gave HELO/EHLO as our IP address log_message = HELO/EHLO our IP address condition = ${if eq {$sender_helo_name}{$interface_address} {yes}{no}}
accept
NOTE: Pursue HELO checking at your own peril. The HELO is fairly unimportant in the grand scheme of SMTP these days, so don't put too much faith in whatever it contains. Some spam might seem to use a telltale HELO string, but you might be surprised at how many legitimate messages start off with a questionable HELO as well. Anyway, it's just as easy for a spammer to send a proper HELO than it is to send HELO im.a.spammer, so consider yourself lucky if you're able to stop much spam this way.
Next, you can perform a check on the sender address or remote host. This shows how to do that after the RCPT TO command; if you reject here, as opposed to rejecting after the MAIL FROM, you'll have better data to log, such as who the message was intended for.
# Specify the ACL to use after RCPT TOacl_smtp_rcpt = check_recipient
# Conditions for the check_recipient ACLcheck_recipient:
# [...]
drop hosts = /etc/exim_reject_hosts drop senders = /etc/exim_reject_senders
# [ Probably a whole lot more... ]
This example uses two plain text files as blacklists. Add appropriate entries to these files - hostnames/IP addresses to /etc/exim_reject_hosts, addresses to /etc/exim_reject_senders, one entry per line.
It is also possible to perform content scanning using a regex against the body of a message, though obviously this can cause Exim to use more CPU than it otherwise would need to, especially on large messages.
# Specify the ACL to use after DATAacl_smtp_data = check_message
# Conditions for the check_messages ACLcheck_message:
deny message = "Sorry, Charlie: $regex_match_string" regex = ^Subject:: .*Lower your self-esteem by becoming a sysadmin
accept
Fix SMTP-Auth for Pine
If pine can't use SMTP authentication on an Exim host and just returns an "unable to authenticate" message without even asking for a password, add the following line to exim.conf:
begin authenticators
fixed_plain: driver = plaintext public_name = PLAIN server_condition = "${perl{checkuserpass}{$1}{$2}{$3}}" server_set_id = $2> server_prompts = :
This was a problem on CPanel Exim builds awhile ago, but they seem to have added this line to their current stock configuration.Log the subject line
This is one of the most useful configuration tweaks I've ever found for Exim. Add this to exim.conf, and you can log the subject lines of messages that pass through your server. This is great for troubleshooting, and for getting a very rough idea of what messages may be spam.
log_selector = +subject
Reducing or increasing what is logged.Disable identd lookups
Frankly, I don't think identd has been useful for a long time, if ever. Identd relies on the connecting host to confirm the identity (system UID) of the remote user who owns the process that is making the network connection. This may be of some use in the world of shell accounts and IRC users, but it really has no place on a high-volume SMTP server, where the UID is often simply "mail" or whatever the remote MTA runs as, which is useless to know. It's overhead, and results in nothing but delays while the identd query is refused or times out. You can stop your Exim server from making these queries by setting the timeout to zero seconds in exim.conf:
rfc1413_query_timeout = 0s
Disable Attachment Blocking
To disable the executable-attachment blocking that many Cpanel servers do by default but don't provide any controls for on a per-domain basis, add the following block to the beginning of the /etc/antivirus.exim file:
if $header_to: matches "example\.comexample2\.com"then finishendif
It is probably possible to use a separate file to list these domains, but I haven't had to do this enough times to warrant setting such a thing up.Searching the logs with exigrep
The exigrep utility (not to be confused with exiqgrep) is used to search an exim log for a string or pattern. It will print all log entries with the same internal message-id as those that matched the pattern, which is very handy since any message will take up at least three lines in the log. exigrep will search the entire content of a log entry, not just particular fields.
One can search for messages sent from a particular IP address:
root@localhost# exigrep '<= .* \[12.34.56.78\] ' /path/to/exim_log
Search for messages sent to a particular IP address:
root@localhost# exigrep '=> .* \[12.34.56.78\]' /path/to/exim_log
This example searches for outgoing messages, which have the "=>" symbol, sent to "user@domain.tld". The pipe to grep for the "<=" symbol will match only the lines with information on the sender - the From address, the sender's IP address, the message size, the message ID, and the subject line if you have enabled logging the subject. The purpose of doing such a search is that the desired information is not on the same log line as the string being searched for.
root@localhost# exigrep '=> .*user@domain.tld' /path/to/exim_log fgrep '<='
Generate and display Exim stats from a logfile:
root@localhost# eximstats /path/to/exim_mainlog
Same as above, with less verbose output:
root@localhost# eximstats -ne -nr -nt /path/to/exim_mainlog
Same as above, for one particular day:
root@localhost# fgrep YYYY-MM-DD /path/to/exim_mainlog eximstats
Bonus!
To delete all queued messages containing a certain string in the body:
root@localhost# grep -lr 'a certain string' /var/spool/exim/input/ \ sed -e 's/^.*\/\([a-zA-Z0-9-]*\)-[DH]$/\1/g' xargs exim -Mrm
Note that the above only delves into /var/spool/exim in order to grep for queue files with the given string, and that's just because exiqgrep doesn't have a feature to grep the actual bodies of messages. If you are deleting these files directly, YOU ARE DOING IT WRONG! Use the appropriate exim command to properly deal with the queue.
If you have to feed many, many message-ids (such as the output of an `exiqgrep -i` command that returns a lot of matches) to an exim command, you may exhaust the limit of your shell's command line arguments. In that case, pipe the listing of message-ids into xargs to run only a limited number of them at once. For example, to remove thousands of messages sent from joe@example.com:
root@localhost# exiqgrep -i -f '<'">joe@example.com>' xargs exim -Mrm
Speaking of "DOING IT WRONG" -- Attention, CPanel forum readers
I get a number of hits to this page from a link in this post at the CPanel forums. The question is:
Due to spamming, spoofing from fields, etc., etc., etc., I am finding it necessary to spend more time to clear the exim queue from time to time. [...] what command would I use to delete the queue
The answer is: Just turn exim off, because your customers are better off knowing that email simply isn't running on your server, than having their queued messages deleted without notice.
Or, figure out what is happening. The examples given in that post pay no regard to the legitimacy of any message, they simply delete everything, making the presumption that if a message is in the queue, it's junk. That is total fallacy. There are a number of reasons legitimate mail can end up in the queue. Maybe your backups or CPanel's "upcp" process are running, and your load average is high -- exim goes into a queue-only mode at a certain threshold, where it stops trying to deliver messages as they come in and just queues them until the load goes back down. Or, maybe it's an outgoing message, and the DNS lookup failed, or the connection to the domain's MX failed, or maybe the remote MX is busy or greylisting you with a 4xx deferral. These are all temporary failures, not permanent ones, and the whole point of having temporary failures in SMTP and a mail queue in your MTA is to be able to try again after awhile.
Exim already purges messages from the queue after the period of time specified in exim.conf. If you have this value set appropriately, there is absolutely no point in removing everything from your queue every day with a cron job. You will lose legitimate mail, and the sender and recipient will never know if or why it happened. Do not do this!
If you regularly have a large number of messages in your queue, find out why they are there. If they are outbound messages, see who is sending them, where they're addressed to, and why they aren't getting there. If they are inbound messages, find out why they aren't getting delivered to your user's account. If you need to delete some, use exiqgrep to pick out just the ones that should be deleted.Reload the configuration
After making changes to exim.conf, you need to give the main exim pid a SIGHUP to re-exec it and have the configuration re-read. Sure, you could stop and start the service, but that's overkill and causes a few seconds of unnecessary downtime. Just do this:
root@localhost# kill -HUP `cat /var/spool/exim/exim-daemon.pid`
You should then see something resembling the following in exim_mainlog:
pid 1079: SIGHUP received: re-exec daemonexim 4.52 daemon started: pid=1079, -q1h, listening for SMTP on port 25 (IPv4)
EXIM
EXIM
What is Exim?
Well this is Exim defined by their webpage (http://www.exim.org)
“Exim is a mail transfer agent (MTA) for hosts that are running Unix or Unix-like operating systems. It was designed on the assumption that it would be run on hosts that are permanently connected to the Internet.”
Basically Exim takes email in to your server and stores it in a mailbox for you to retrieve at a later date. It also takes email you send and sends it to another server.
About this documentation:
The domain we will use as an example is: mydomain.com
How does it know how to deliver mail?
Well first it reads through a configuration file (/etc/exim.conf) and gets rules from there. These rules tell it how to handle the file. One of the rules tells it to check (/etc/valiases/domainname.tld), the valias file.
What is a valias file and how does it work?
Well this is a simple valias file:
Quote:
ajz@mydomain.com: ajz@otherdomain.com
ajz@mydomain.com: ajz@otherdomain.net
*: :blackhole:
Lets take a look at the first line “ajz@mydomain.com: ajz@otherdomain.com”
This takes any email to ajz@mydomain.com and forwards it to ajz@otherdomain.com
The second line is very similar “ajz@mydomain.com: ajz@otherdomain.net”
This takes any email send to ajz@mydomain.com and forwards it to "ajz@otherdomain.net"
Now as you can see it goes from top to bottom. If it cant find a specific email address it goes to the last line *: :blackhole: in this case. Now, * means anything else. The :blackhole: is a special system call, basically it sends everything to /dev/null also known as deleted.
So anything else sent to mydomain.com will be immediately deleted.
The other option besides :blackhole: is :fail:. This send a bounce back email saying that the email could not be sent.
You can use also :blackhole: with any other email address also (like xyz@mydomain.com : :blackhole
What is the config file and how does it work?
The config file is located at /etc/exim.conf and has many parts to it (you can also see this in WHM under Service Configuration -> Exim Configuration Editor). It is how Exim separates and sends email. Now let us take a look at the default config file that comes with cPanel.
Code:
#!!# cPanel Exim 4 Config
Well this is just as it looks, the comment letting us know that this is cPanel’s Exim config file.
After this comes a text box, you can put some configuration options. This is always referred to as “Exim box 1”.
Code:
acl_smtp_rcpt = check_recipient
This option defines the ACL that is run when an SMTP RCPT command is received. Here it runs a separate part of code that is defined later as a sub routine or ACL. This sub routine is the check_recipient routine.
Code:
acl_smtp_data = check_message
This option defines the ACL that is run after an SMTP DATA command has been processed and the message itself has been received, but before the final acknowledgement is sent.
In this code the ACL that is ran is check_message
Code:
domainlist local_domains = lsearch;/etc/localdomains
This setting defines a named domain list called local_domains, created from the old options that referred to local domains. It will be referenced later on by the syntax "+local_domains".
Basically it open /etc/localdomains and reads each line as a separate “local domain” so that it can be called later as +local_domains
Code:
domainlist relay_domains = lsearch;/etc/localdomains : \
lsearch;/etc/secondarymx
This defines +relay_domains the same way as +local_domains, but this time it also includes /etc/secondarymx
Code:
hostlist relay_hosts = lsearch;/etc/relayhosts : \
localhost
Once again this defines +relay_hosts from /etc/relayhosts and also adds in the localhost hostname.
Code:
hostlist auth_relay_hosts = *
This sets +auth_relay_hosts as * or all
Now lets look more into the MAIN CONFIGURATION SETTINGS
Code:
perl_startup = do '/etc/exim.pl'
This runs the perl script /etc/exim.pl on startup.
Code:
smtp_banner = "${primary_hostname} ESMTP Exim ${version_number} \
\#${compile_number} ${tod_full} \n\
We do not authorize the use of this system to transport unsolicited, \n\
and/or bulk e-mail."
This shows the Exim login from smtp. To test it out “telnet mydomain.com 110” from a dos prompt. You should see what is there (with variables replaces).
Code:
no_local_from_check
Prevents user ‘nobody’ from sending email (this is set under tweak settings)
Code:
rfc1413_query_timeout = 2s
This sets the timeout on RFC 1413 identification calls. If it is set to zero, no RFC 1413 calls are ever made.
Taken from the RFC Documentation about what RFC 1413 (http://www.faqs.org/rfcs/rfc1413.html) is:
“This is a connection based application on TCP. A server listens for TCP connections on TCP port 113 (decimal). Once a connection is established, the server reads a line of data which specifies the connection of interest. If it exists, the system dependent user identifier of the connection of interest is sent as the reply. The server may then either shut the connection down or it may continue to read/respond to multiple queries.”
Code:
smtp_connect_backlog = 50
This option specifies a maximum number of waiting SMTP connections. Exim passes this value to the TCP/IP system when it sets up its listener. Once this number of connections are waiting for the daemon's attention, subsequent connection attempts are refused at the TCP/IP level. At least, that is what the manuals say; in some circumstances such connection attempts have been observed to time out instead. For large systems it is probably a good idea to increase the value (to 50, say). It also gives some protection against denial-of-service attacks by SYN flooding.
Code:
smtp_accept_max = 100
This option specifies the maximum number of simultaneous incoming SMTP calls that Exim will accept. It applies only to the listening daemon; there is no control (in Exim) when incoming SMTP is being handled by inetd. If the value is set to zero, no limit is applied.
Code:
deliver_queue_load_max = 3
When this option is set, a queue run is abandoned if the system load average becomes greater than the value of the option.
Code:
auto_thaw = 1h
If this option is set to a time greater than zero, a queue runner will try a new delivery attempt on any frozen message if this much time has passed since it was frozen. This may result in the message being re-frozen if nothing has changed since the last attempt.
Code:
system_filter = /etc/antivirus.exim
This runs all email through a filter located at /etc/antivirus.exim
Code:
message_body_visible = 5000
This option specifies how much of a message's body is to be included in the $message_body and $message_body_end expansion variables. (will be used later on)
Code:
never_users = root
This is a safety concern so that email is never attempted to be sent as the root user.
Code:
Timeout_frozen_after = 7d
Any email that is frozen after 7d will be removed. (This email is basically undeliverable)
Code:
tls_certificate = /etc/exim.crt
The Cert for secure (TLS) Exim port.
Code:
tls_privatekey = /etc/exim.key
The private key for secure (TLS) Exim port.
Code:
tls_advertise_hosts = *
When Exim is built with support for TLS encrypted connections, the availability of the STARTTLS command to set up an encrypted session is advertised in response to EHLO only to those client hosts that match this option
Makes all domains able to use TLS
Code:
helo_accept_junk_hosts = *
Exim checks the syntax of HELO and EHLO commands for incoming SMTP mail, and gives an error response for invalid data. Unfortunately, there are some SMTP clients that send syntactic junk. They can be accommodated by setting this option.
Code:
smtp_enforce_sync = false
The SMTP protocol specification requires the client to wait for a response from the server at certain points in the dialogue. Without PIPELINING these synchronization points are after every command; with PIPELINING they are fewer, but they still exist. Some spamming sites send out a complete set of SMTP commands without waiting for any response. Exim protects against this by rejecting a message if the client has sent further input when it should not have. The error response “554 SMTP synchronization error” is sent, and the connection is dropped. Testing for this error cannot be perfect because of transmission delays (unexpected input may be on its way but not yet received when Exim checks).
Code:
begin acl
This lets Exim know that we will begin making the ACL sub routines (the ones we defined above).
Next we have Exim Box 2 (this should be empty by default)
Then we have Exim Box 3, this will have a lot of text in it (all of the ACL’s)
Code:
accept hosts = :
This starts the accept_hosts calls
Code:
warn message = X-WhitelistedRCPT-nohdrfromcallback: Yes
condition = \
${if and {{match{$local_part}{(.*)-bounces\+.*}} \
{exists {/usr/local/cpanel/3rdparty/mailman/lists/${lc:$1}/config.pck}}} \
{yes}{no}}
accept condition = \
${if and {{match{$local_part}{(.*)-bounces\+.*}} \
{exists {/usr/local/cpanel/3rdparty/mailman/lists/${lc:$1}/config.pck}}} \
{yes}{no}}
This accepts bounces to lists even if callbacks or other checks would fail
Code:
require verify = sender
accept domains = +local_domains
endpass
This verifies all message senders that are not sent to a mailman list and if it can’t verify the sender it bounces it.
Code:
message = "The recipient cannot be verified. Please check all recipients of this message to verify they are valid."
verify = recipient
This verifies the recipient, if it cannot it bounces it and says “The recipient cannot be verified. Please check all recipients of this message to verify they are valid.”
Code:
accept domains = +relay_domains
Accepts domains that are in +relay_domains (that we set earlier)
Code:
warn message = ${perl{popbeforesmtpwarn}{$sender_host_name}}
hosts = +relay_hosts
accept hosts = +relay_hosts
Checks to see if the sending host name is in +relay_hosts(we set this earlier) and if it is not it run it through the perl script (we also sent earlier)
Code:
warn message = ${perl{popbeforesmtpwarn}{$sender_host_address}}
condition = ${perl{checkrelayhost}{$sender_host_address}}
accept condition = ${perl{checkrelayhost}{$sender_host_address}}
Runs the user host address through the perl script.
Code:
accept hosts = +auth_relay_hosts
endpass
message = $sender_fullhost is currently not permitted to \
relay through this server. Perhaps you \
have not logged into the pop/imap server in the \
last 30 minutes or do not have SMTP Authentication turned on in your email client.
authenticated = *
Accepts hosts that are in +auth_relay_hosts or if they are authenticated otherwise it errors out.
Code:
deny message = $sender_fullhost is currently not permitted to \
relay through this server. Perhaps you \
have not logged into the pop/imap server in the \
last 30 minutes or do not have SMTP Authentication turned on in your email client.
If they are denied it errors with this message
Code:
check_message:
require verify = header_sender
accept
Checks the message and makes sure it has correct headers.
That is the end of this box!
Code:
begin authenticators
fixed_plain:
driver = plaintext
public_name = PLAIN
server_condition = "${perl{checkuserpass}{$1}{$2}{$3}}"
server_set_id = $2
fixed_login:
driver = plaintext
public_name = LOGIN
server_prompts = "Username:: : Password::"
server_condition = "${perl{checkuserpass}{$1}{$2}}"
server_set_id = $1
Does the login authentication for SMTP.
This is followed by another blank box (if you need it)
Code:
begin rewrite
Begins the rewrite section
Code:
nobody@lsearch;/etc/localdomains "${if !eq {$header_From:}{}{$header_sender:$header_From:}fail}" Fs
cpanel@lsearch;/etc/localdomains "${if !eq {$header_From:}{}{$header_sender:$header_From:}fail}" Fs
Tries to rewrite the cpanel & nobody to the domain name instead of server name.
Followed by another blank box (for more rewrites)
Code:
begin routers
Lets Exim know you are beginning the routers (how Exim handles non-local email)
Code:
mailman_virtual_router:
driver = accept
require_files = /usr/local/cpanel/3rdparty/mailman/lists/${lc::$local_part}_${lc::$domain}/config.pck
local_part_suffix_optional
local_part_suffix = -admin : \
-bounces : -bounces+* : \
-confirm : -confirm+* : \
-join : -leave : \
-owner : -request : \
-subscribe : -unsubscribe
transport = mailman_virtual_transport
mailman_virtual_router_nodns:
driver = accept
require_files = /usr/local/cpanel/3rdparty/mailman/lists/${lc::$local_part}/config.pck
condition = \
${if or {{match{$local_part}{.*_.*}} \
{eq{$local_part}{mailman}}} \
{1}{0}}
local_part_suffix_optional
local_part_suffix = -admin : \
-bounces : -bounces+* : \
-confirm : -confirm+* : \
-join : -leave : \
-owner : -request : \
-subscribe : -unsubscribe
transport = mailman_virtual_transport_nodns
If we are trying to deliver to a remote mailman domain that is on the localhost let it go though even if its not in /etc/localdomains since mailman will eat up 100% of the cpu if we don't
Next we have another Blank box (for routers this time)
Code:
lookuphost:
driver = dnslookup
condition = "${perl{checkspam}}"
domains = ! +local_domains
#ignore verisign to prevent waste of bandwidth
ignore_target_hosts = 0.0.0.0 : 127.0.0.0/8 : 64.94.110.0/24
headers_add = "${perl{mailtrapheaders}}"
transport = remote_smtp
This router routes to remote hosts over SMTP using a DNS lookup with default options.
Code:
literal:
driver = ipliteral
condition = "${perl{checkspam}}"
domains = ! +local_domains
headers_add = "${perl{mailtrapheaders}}"
transport = remote_smtp
This router routes to remote hosts over SMTP by explicit IP address, given as a "domain literal" in the form [nnn.nnn.nnn.nnn]. The RFCs require this facility, which is why it is enabled by default in Exim. If you want to lock it out, set forbid_domain_literals in the main configuration section above.
Code:
fail_remote_domains:
driver = redirect
domains = ! +local_domains
allow_fail
data = :fail: unrouteable mail domain "$domain"
This new router is put here to fail all domains that were not in local_domains in the Exim 3 configuration.
Another Blank Box
DIRECTORS CONFIGURATION (Specifies how local addresses are handled)
Code:
virtual_sa_user:
driver = accept
headers_add="${perl{gensaheader_virtual}{$domain}}"
condition = "${perl{checksa_deliver}{$domain}{$local_part}{$received_protocol}}"
domains = lsearch;/etc/userdomains
retry_use_local_part
transport = virtual_sa_userdelivery
sa_localuser:
driver = accept
check_local_user
headers_add="${perl{gensaheader}{$local_part}}"
condition = "${perl{checkusersa}{$local_part}{$received_protocol}}"
domains = ! lsearch;/etc/userdomains
transport = local_sa_delivery
central_filter:
#!!# filter renamed allow_filter
driver = redirect
allow_filter
no_check_local_user
file = /etc/vfilters/${domain}
file_transport = address_file
pipe_transport = virtual_address_pipe
reply_transport = address_reply
retry_use_local_part
user = "${lookup{$domain}lsearch* {/etc/userdomains}{$value}}"
no_verify
central_user_filter:
driver = redirect
allow_filter
check_local_user
domains = ! lsearch;/etc/userdomains
condition = "${perl{hasfilterfile}{$local_part}}"
file = "${perl{getfilterfile}{$local_part}}"
file_transport = address_file
pipe_transport = virtual_address_pipe
reply_transport = address_reply
retry_use_local_part
no_verify
virtual_aliases_nostar:
driver = redirect
allow_defer
allow_fail
data = ${if exists{/etc/valiases/$domain}{${lookup{$local_part@$domain}lsearch{/etc/valiases/$domain}}}}
file_transport = address_file
group = mail
pipe_transport = virtual_address_pipe
retry_use_local_part
domains = lsearch;/etc/localdomains
unseen
virtual_user_spam:
driver = accept
condition = "${perl{check_deliver_spam}{$domain}{$local_part}}"
headers_remove="x-spam-exim"
domains = lsearch;/etc/userdomains
retry_use_local_part
transport = virtual_userdelivery_spam
virtual_user:
driver = accept
condition = "${perl{check_deliver}{$domain}{$local_part}}"
headers_remove="x-spam-exim"
domains = lsearch;/etc/userdomains
retry_use_local_part
transport = virtual_userdelivery
has_alias_but_no_mailbox_discarded_to_prevent_loop:
driver = redirect
condition = "${perl{checkvalias}{$domain}{$local_part}}"
domains = lsearch;/etc/localdomains
data="#Exim Filter\nseen finish"
group = "${lookup{$domain}lsearch* {/etc/userdomains}{$value}}"
user = "${lookup{$domain}lsearch* {/etc/userdomains}{$value}}"
allow_filter
disable_logging = true
virtual_aliases:
driver = redirect
allow_defer
allow_fail
data = ${if exists{/etc/valiases/$domain}{${lookup{*}lsearch{/etc/valiases/$domain}}}}
file_transport = address_file
group = mail
pipe_transport = virtual_address_pipe
domains = lsearch;/etc/localdomains
retry_use_local_part
Verifies that the user exists on the server and if so puts the email in that mail box (I grouped them all together because they are all pretty much the same)
Code:
system_aliases:
driver = redirect
allow_defer
allow_fail
data = ${lookup{$local_part}lsearch{/etc/aliases}}
file_transport = address_file
pipe_transport = address_pipe
retry_use_local_part
# user = exim
local_aliases:
driver = redirect
allow_defer
allow_fail
data = ${lookup{$local_part}lsearch{/etc/localaliases}}
file_transport = address_file
pipe_transport = address_pipe
check_local_user
userforward:
#!!# filter renamed allow_filter
driver = redirect
allow_filter
check_ancestor
check_local_user
domains = ! lsearch;/etc/userdomains
no_expn
file = $home/.forward
file_transport = address_file
pipe_transport = address_pipe
reply_transport = address_reply
no_verify
localuser_spam:
driver = accept
headers_remove="x-spam-exim"
condition = "${perl{checkuserspambox}{$local_part}}"
check_local_user
domains = ! lsearch;/etc/userdomains
transport = local_delivery_spam
localuser:
driver = accept
headers_remove="x-spam-exim"
check_local_user
domains = ! lsearch;/etc/userdomains
transport = local_delivery
Allows the .forward that some people like
Another Box
TRANSPORTS CONFIGURATION (the configurations of the transports used by exim)
Code:
begin transports
Tells Exim that we define the transports here
Code:
remote_smtp:
driver = smtp
This is straight forward, the remote_smtp transport uses the smtp protocol
Code:
local_delivery:
driver = appendfile
delivery_date_add
envelope_to_add
file = "${extract{5}{:}{${lookup{$local_part}lsearch{/etc/passwd}{$value}}}}/mail/inbox"
group = mail
mode = 0660
return_path_add
user = $local_part
Normal email is sent to the mail/inbox folder for the user
Code:
local_delivery_spam:
driver = appendfile
delivery_date_add
envelope_to_add
file = "${extract{5}{:}{${lookup{$local_part}lsearch{/etc/passwd}{$value}}}}/mail/spam"
group = mail
mode = 0660
return_path_add
user = $local_part
This sends email marked as spam to the mail/spam folder
Code:
local_sa_delivery:
driver = pipe
command = /usr/sbin/sendmail -bS
use_bsmtp = true
transport_filter = "/usr/bin/spamc"
user = $local_part
group = mail
log_output = true
current_directory = "/tmp"
home_directory = "/tmp"
return_fail_output = true
return_path_add = false
message_prefix =
message_suffix =
Sends the email through spamassasin
Code:
address_pipe:
driver = pipe
return_output
This pipes email through
Code:
virtual_address_pipe:
driver = pipe
group = "${lookup{$domain}lsearch* {/etc/userdomains}{$value}}"
return_output
user = "${lookup{$domain}lsearch* {/etc/userdomains}{$value}}"
Also just helps with piping the email through to the location request in the valias.
Code:
address_file:
driver = appendfile
delivery_date_add
envelope_to_add
return_path_add
This transport is used for handling deliveries directly to files that are generated by aliasing or forwarding.
Code:
virtual_sa_userdelivery:
driver = pipe
command = /usr/sbin/sendmail -bS
use_bsmtp = true
transport_filter = "/usr/bin/spamc"
user = "${lookup{$domain}lsearch* {/etc/userdomains}{$value}}"
group = mail
log_output = true
current_directory = "/tmp"
home_directory = "/tmp"
return_fail_output = true
return_path_add = false
message_prefix =
message_suffix =
virtual_userdelivery_spam:
driver = appendfile
delivery_date_add
envelope_to_add
file = "${extract{5}{:}{${lookup{${lookup{$domain}lsearch*{/etc/userdomains}{$value}}}lsearch{/etc/passwd}{$value}}}}/mail/${domain}/${local_part}/spam"
group = mail
mode = 0660
quota = "${if exists{${extract{5}{:}{${lookup{${lookup{$domain}lsearch*{/etc/userdomains}{$value}}}lsearch{/etc/passwd}{$value}}}}/etc/${domain}/quota} {${lookup{$local_part}lsearch*{${extract{5}{:}{${lookup{${lookup{$domain}lsearch*{/etc/userdomains}{$value}}}lsearch{/etc/passwd}{$value}}}}/etc/${domain}/quota}{$value}}} {}}"
return_path_add
user = "${lookup{$domain}lsearch* {/etc/userdomains}{$value}}"
virtual_userdelivery:
driver = appendfile
delivery_date_add
envelope_to_add
file = "${extract{5}{:}{${lookup{${lookup{$domain}lsearch*{/etc/userdomains}{$value}}}lsearch{/etc/passwd}{$value}}}}/mail/${domain}/${local_part}/inbox"
group = mail
mode = 0660
quota = "${if exists{${extract{5}{:}{${lookup{${lookup{$domain}lsearch*{/etc/userdomains}{$value}}}lsearch{/etc/passwd}{$value}}}}/etc/${domain}/quota} {${lookup{$local_part}lsearch*{${extract{5}{:}{${lookup{${lookup{$domain}lsearch*{/etc/userdomains}{$value}}}lsearch{/etc/passwd}{$value}}}}/etc/${domain}/quota}{$value}}} {}}"
return_path_add
user = "${lookup{$domain}lsearch* {/etc/userdomains}{$value}}"
This transport is used for handling autoreplies generated by the filtering option of the forwardfile director.
Code:
address_reply:
driver = autoreply
This is for autoreplies
Code:
mailman_virtual_transport:
driver = pipe
command = /usr/local/cpanel/3rdparty/mailman/mail/mailman \
'${if def:local_part_suffix \
{${sg{$local_part_suffix}{-(\\w+)(\\+.*)?}{\$1}}} \
{post}}' \
${lc:$local_part}_${lc:$domain}
current_directory = /usr/local/cpanel/3rdparty/mailman
home_directory = /usr/local/cpanel/3rdparty/mailman
user = mailman
group = mailman
mailman_virtual_transport_nodns:
driver = pipe
command = /usr/local/cpanel/3rdparty/mailman/mail/mailman \
'${if def:local_part_suffix \
{${sg{$local_part_suffix}{-(\\w+)(\\+.*)?}{\$1}}} \
{post}}' \
${lc:$local_part}
current_directory = /usr/local/cpanel/3rdparty/mailman
home_directory = /usr/local/cpanel/3rdparty/mailman
user = mailman
group = mailman
Mailman transports
RETRY CONFIGURATION
Code:
begin retry
Tells Exim that we are starting the retry rules
Code:
* * F,2h,15m; G,16h,1h,1.5; F,4d,8h
This single retry rule applies to all domains and all errors. It specifies retries every 15 minutes for 2 hours, then increasing retry intervals, starting at 1 hour and increasing each time by a factor of 1.5, up to 16 hours, then retries every 8 hours until 4 days have passed since the first failed delivery.
That is your Exim config files and how Exim works on your cPanel box.
What is Exim?
Well this is Exim defined by their webpage (http://www.exim.org)
“Exim is a mail transfer agent (MTA) for hosts that are running Unix or Unix-like operating systems. It was designed on the assumption that it would be run on hosts that are permanently connected to the Internet.”
Basically Exim takes email in to your server and stores it in a mailbox for you to retrieve at a later date. It also takes email you send and sends it to another server.
About this documentation:
The domain we will use as an example is: mydomain.com
How does it know how to deliver mail?
Well first it reads through a configuration file (/etc/exim.conf) and gets rules from there. These rules tell it how to handle the file. One of the rules tells it to check (/etc/valiases/domainname.tld), the valias file.
What is a valias file and how does it work?
Well this is a simple valias file:
Quote:
ajz@mydomain.com: ajz@otherdomain.com
ajz@mydomain.com: ajz@otherdomain.net
*: :blackhole:
Lets take a look at the first line “ajz@mydomain.com: ajz@otherdomain.com”
This takes any email to ajz@mydomain.com and forwards it to ajz@otherdomain.com
The second line is very similar “ajz@mydomain.com: ajz@otherdomain.net”
This takes any email send to ajz@mydomain.com and forwards it to "ajz@otherdomain.net"
Now as you can see it goes from top to bottom. If it cant find a specific email address it goes to the last line *: :blackhole: in this case. Now, * means anything else. The :blackhole: is a special system call, basically it sends everything to /dev/null also known as deleted.
So anything else sent to mydomain.com will be immediately deleted.
The other option besides :blackhole: is :fail:. This send a bounce back email saying that the email could not be sent.
You can use also :blackhole: with any other email address also (like xyz@mydomain.com : :blackhole
What is the config file and how does it work?
The config file is located at /etc/exim.conf and has many parts to it (you can also see this in WHM under Service Configuration -> Exim Configuration Editor). It is how Exim separates and sends email. Now let us take a look at the default config file that comes with cPanel.
Code:
#!!# cPanel Exim 4 Config
Well this is just as it looks, the comment letting us know that this is cPanel’s Exim config file.
After this comes a text box, you can put some configuration options. This is always referred to as “Exim box 1”.
Code:
acl_smtp_rcpt = check_recipient
This option defines the ACL that is run when an SMTP RCPT command is received. Here it runs a separate part of code that is defined later as a sub routine or ACL. This sub routine is the check_recipient routine.
Code:
acl_smtp_data = check_message
This option defines the ACL that is run after an SMTP DATA command has been processed and the message itself has been received, but before the final acknowledgement is sent.
In this code the ACL that is ran is check_message
Code:
domainlist local_domains = lsearch;/etc/localdomains
This setting defines a named domain list called local_domains, created from the old options that referred to local domains. It will be referenced later on by the syntax "+local_domains".
Basically it open /etc/localdomains and reads each line as a separate “local domain” so that it can be called later as +local_domains
Code:
domainlist relay_domains = lsearch;/etc/localdomains : \
lsearch;/etc/secondarymx
This defines +relay_domains the same way as +local_domains, but this time it also includes /etc/secondarymx
Code:
hostlist relay_hosts = lsearch;/etc/relayhosts : \
localhost
Once again this defines +relay_hosts from /etc/relayhosts and also adds in the localhost hostname.
Code:
hostlist auth_relay_hosts = *
This sets +auth_relay_hosts as * or all
Now lets look more into the MAIN CONFIGURATION SETTINGS
Code:
perl_startup = do '/etc/exim.pl'
This runs the perl script /etc/exim.pl on startup.
Code:
smtp_banner = "${primary_hostname} ESMTP Exim ${version_number} \
\#${compile_number} ${tod_full} \n\
We do not authorize the use of this system to transport unsolicited, \n\
and/or bulk e-mail."
This shows the Exim login from smtp. To test it out “telnet mydomain.com 110” from a dos prompt. You should see what is there (with variables replaces).
Code:
no_local_from_check
Prevents user ‘nobody’ from sending email (this is set under tweak settings)
Code:
rfc1413_query_timeout = 2s
This sets the timeout on RFC 1413 identification calls. If it is set to zero, no RFC 1413 calls are ever made.
Taken from the RFC Documentation about what RFC 1413 (http://www.faqs.org/rfcs/rfc1413.html) is:
“This is a connection based application on TCP. A server listens for TCP connections on TCP port 113 (decimal). Once a connection is established, the server reads a line of data which specifies the connection of interest. If it exists, the system dependent user identifier of the connection of interest is sent as the reply. The server may then either shut the connection down or it may continue to read/respond to multiple queries.”
Code:
smtp_connect_backlog = 50
This option specifies a maximum number of waiting SMTP connections. Exim passes this value to the TCP/IP system when it sets up its listener. Once this number of connections are waiting for the daemon's attention, subsequent connection attempts are refused at the TCP/IP level. At least, that is what the manuals say; in some circumstances such connection attempts have been observed to time out instead. For large systems it is probably a good idea to increase the value (to 50, say). It also gives some protection against denial-of-service attacks by SYN flooding.
Code:
smtp_accept_max = 100
This option specifies the maximum number of simultaneous incoming SMTP calls that Exim will accept. It applies only to the listening daemon; there is no control (in Exim) when incoming SMTP is being handled by inetd. If the value is set to zero, no limit is applied.
Code:
deliver_queue_load_max = 3
When this option is set, a queue run is abandoned if the system load average becomes greater than the value of the option.
Code:
auto_thaw = 1h
If this option is set to a time greater than zero, a queue runner will try a new delivery attempt on any frozen message if this much time has passed since it was frozen. This may result in the message being re-frozen if nothing has changed since the last attempt.
Code:
system_filter = /etc/antivirus.exim
This runs all email through a filter located at /etc/antivirus.exim
Code:
message_body_visible = 5000
This option specifies how much of a message's body is to be included in the $message_body and $message_body_end expansion variables. (will be used later on)
Code:
never_users = root
This is a safety concern so that email is never attempted to be sent as the root user.
Code:
Timeout_frozen_after = 7d
Any email that is frozen after 7d will be removed. (This email is basically undeliverable)
Code:
tls_certificate = /etc/exim.crt
The Cert for secure (TLS) Exim port.
Code:
tls_privatekey = /etc/exim.key
The private key for secure (TLS) Exim port.
Code:
tls_advertise_hosts = *
When Exim is built with support for TLS encrypted connections, the availability of the STARTTLS command to set up an encrypted session is advertised in response to EHLO only to those client hosts that match this option
Makes all domains able to use TLS
Code:
helo_accept_junk_hosts = *
Exim checks the syntax of HELO and EHLO commands for incoming SMTP mail, and gives an error response for invalid data. Unfortunately, there are some SMTP clients that send syntactic junk. They can be accommodated by setting this option.
Code:
smtp_enforce_sync = false
The SMTP protocol specification requires the client to wait for a response from the server at certain points in the dialogue. Without PIPELINING these synchronization points are after every command; with PIPELINING they are fewer, but they still exist. Some spamming sites send out a complete set of SMTP commands without waiting for any response. Exim protects against this by rejecting a message if the client has sent further input when it should not have. The error response “554 SMTP synchronization error” is sent, and the connection is dropped. Testing for this error cannot be perfect because of transmission delays (unexpected input may be on its way but not yet received when Exim checks).
Code:
begin acl
This lets Exim know that we will begin making the ACL sub routines (the ones we defined above).
Next we have Exim Box 2 (this should be empty by default)
Then we have Exim Box 3, this will have a lot of text in it (all of the ACL’s)
Code:
accept hosts = :
This starts the accept_hosts calls
Code:
warn message = X-WhitelistedRCPT-nohdrfromcallback: Yes
condition = \
${if and {{match{$local_part}{(.*)-bounces\+.*}} \
{exists {/usr/local/cpanel/3rdparty/mailman/lists/${lc:$1}/config.pck}}} \
{yes}{no}}
accept condition = \
${if and {{match{$local_part}{(.*)-bounces\+.*}} \
{exists {/usr/local/cpanel/3rdparty/mailman/lists/${lc:$1}/config.pck}}} \
{yes}{no}}
This accepts bounces to lists even if callbacks or other checks would fail
Code:
require verify = sender
accept domains = +local_domains
endpass
This verifies all message senders that are not sent to a mailman list and if it can’t verify the sender it bounces it.
Code:
message = "The recipient cannot be verified. Please check all recipients of this message to verify they are valid."
verify = recipient
This verifies the recipient, if it cannot it bounces it and says “The recipient cannot be verified. Please check all recipients of this message to verify they are valid.”
Code:
accept domains = +relay_domains
Accepts domains that are in +relay_domains (that we set earlier)
Code:
warn message = ${perl{popbeforesmtpwarn}{$sender_host_name}}
hosts = +relay_hosts
accept hosts = +relay_hosts
Checks to see if the sending host name is in +relay_hosts(we set this earlier) and if it is not it run it through the perl script (we also sent earlier)
Code:
warn message = ${perl{popbeforesmtpwarn}{$sender_host_address}}
condition = ${perl{checkrelayhost}{$sender_host_address}}
accept condition = ${perl{checkrelayhost}{$sender_host_address}}
Runs the user host address through the perl script.
Code:
accept hosts = +auth_relay_hosts
endpass
message = $sender_fullhost is currently not permitted to \
relay through this server. Perhaps you \
have not logged into the pop/imap server in the \
last 30 minutes or do not have SMTP Authentication turned on in your email client.
authenticated = *
Accepts hosts that are in +auth_relay_hosts or if they are authenticated otherwise it errors out.
Code:
deny message = $sender_fullhost is currently not permitted to \
relay through this server. Perhaps you \
have not logged into the pop/imap server in the \
last 30 minutes or do not have SMTP Authentication turned on in your email client.
If they are denied it errors with this message
Code:
check_message:
require verify = header_sender
accept
Checks the message and makes sure it has correct headers.
That is the end of this box!
Code:
begin authenticators
fixed_plain:
driver = plaintext
public_name = PLAIN
server_condition = "${perl{checkuserpass}{$1}{$2}{$3}}"
server_set_id = $2
fixed_login:
driver = plaintext
public_name = LOGIN
server_prompts = "Username:: : Password::"
server_condition = "${perl{checkuserpass}{$1}{$2}}"
server_set_id = $1
Does the login authentication for SMTP.
This is followed by another blank box (if you need it)
Code:
begin rewrite
Begins the rewrite section
Code:
nobody@lsearch;/etc/localdomains "${if !eq {$header_From:}{}{$header_sender:$header_From:}fail}" Fs
cpanel@lsearch;/etc/localdomains "${if !eq {$header_From:}{}{$header_sender:$header_From:}fail}" Fs
Tries to rewrite the cpanel & nobody to the domain name instead of server name.
Followed by another blank box (for more rewrites)
Code:
begin routers
Lets Exim know you are beginning the routers (how Exim handles non-local email)
Code:
mailman_virtual_router:
driver = accept
require_files = /usr/local/cpanel/3rdparty/mailman/lists/${lc::$local_part}_${lc::$domain}/config.pck
local_part_suffix_optional
local_part_suffix = -admin : \
-bounces : -bounces+* : \
-confirm : -confirm+* : \
-join : -leave : \
-owner : -request : \
-subscribe : -unsubscribe
transport = mailman_virtual_transport
mailman_virtual_router_nodns:
driver = accept
require_files = /usr/local/cpanel/3rdparty/mailman/lists/${lc::$local_part}/config.pck
condition = \
${if or {{match{$local_part}{.*_.*}} \
{eq{$local_part}{mailman}}} \
{1}{0}}
local_part_suffix_optional
local_part_suffix = -admin : \
-bounces : -bounces+* : \
-confirm : -confirm+* : \
-join : -leave : \
-owner : -request : \
-subscribe : -unsubscribe
transport = mailman_virtual_transport_nodns
If we are trying to deliver to a remote mailman domain that is on the localhost let it go though even if its not in /etc/localdomains since mailman will eat up 100% of the cpu if we don't
Next we have another Blank box (for routers this time)
Code:
lookuphost:
driver = dnslookup
condition = "${perl{checkspam}}"
domains = ! +local_domains
#ignore verisign to prevent waste of bandwidth
ignore_target_hosts = 0.0.0.0 : 127.0.0.0/8 : 64.94.110.0/24
headers_add = "${perl{mailtrapheaders}}"
transport = remote_smtp
This router routes to remote hosts over SMTP using a DNS lookup with default options.
Code:
literal:
driver = ipliteral
condition = "${perl{checkspam}}"
domains = ! +local_domains
headers_add = "${perl{mailtrapheaders}}"
transport = remote_smtp
This router routes to remote hosts over SMTP by explicit IP address, given as a "domain literal" in the form [nnn.nnn.nnn.nnn]. The RFCs require this facility, which is why it is enabled by default in Exim. If you want to lock it out, set forbid_domain_literals in the main configuration section above.
Code:
fail_remote_domains:
driver = redirect
domains = ! +local_domains
allow_fail
data = :fail: unrouteable mail domain "$domain"
This new router is put here to fail all domains that were not in local_domains in the Exim 3 configuration.
Another Blank Box
DIRECTORS CONFIGURATION (Specifies how local addresses are handled)
Code:
virtual_sa_user:
driver = accept
headers_add="${perl{gensaheader_virtual}{$domain}}"
condition = "${perl{checksa_deliver}{$domain}{$local_part}{$received_protocol}}"
domains = lsearch;/etc/userdomains
retry_use_local_part
transport = virtual_sa_userdelivery
sa_localuser:
driver = accept
check_local_user
headers_add="${perl{gensaheader}{$local_part}}"
condition = "${perl{checkusersa}{$local_part}{$received_protocol}}"
domains = ! lsearch;/etc/userdomains
transport = local_sa_delivery
central_filter:
#!!# filter renamed allow_filter
driver = redirect
allow_filter
no_check_local_user
file = /etc/vfilters/${domain}
file_transport = address_file
pipe_transport = virtual_address_pipe
reply_transport = address_reply
retry_use_local_part
user = "${lookup{$domain}lsearch* {/etc/userdomains}{$value}}"
no_verify
central_user_filter:
driver = redirect
allow_filter
check_local_user
domains = ! lsearch;/etc/userdomains
condition = "${perl{hasfilterfile}{$local_part}}"
file = "${perl{getfilterfile}{$local_part}}"
file_transport = address_file
pipe_transport = virtual_address_pipe
reply_transport = address_reply
retry_use_local_part
no_verify
virtual_aliases_nostar:
driver = redirect
allow_defer
allow_fail
data = ${if exists{/etc/valiases/$domain}{${lookup{$local_part@$domain}lsearch{/etc/valiases/$domain}}}}
file_transport = address_file
group = mail
pipe_transport = virtual_address_pipe
retry_use_local_part
domains = lsearch;/etc/localdomains
unseen
virtual_user_spam:
driver = accept
condition = "${perl{check_deliver_spam}{$domain}{$local_part}}"
headers_remove="x-spam-exim"
domains = lsearch;/etc/userdomains
retry_use_local_part
transport = virtual_userdelivery_spam
virtual_user:
driver = accept
condition = "${perl{check_deliver}{$domain}{$local_part}}"
headers_remove="x-spam-exim"
domains = lsearch;/etc/userdomains
retry_use_local_part
transport = virtual_userdelivery
has_alias_but_no_mailbox_discarded_to_prevent_loop:
driver = redirect
condition = "${perl{checkvalias}{$domain}{$local_part}}"
domains = lsearch;/etc/localdomains
data="#Exim Filter\nseen finish"
group = "${lookup{$domain}lsearch* {/etc/userdomains}{$value}}"
user = "${lookup{$domain}lsearch* {/etc/userdomains}{$value}}"
allow_filter
disable_logging = true
virtual_aliases:
driver = redirect
allow_defer
allow_fail
data = ${if exists{/etc/valiases/$domain}{${lookup{*}lsearch{/etc/valiases/$domain}}}}
file_transport = address_file
group = mail
pipe_transport = virtual_address_pipe
domains = lsearch;/etc/localdomains
retry_use_local_part
Verifies that the user exists on the server and if so puts the email in that mail box (I grouped them all together because they are all pretty much the same)
Code:
system_aliases:
driver = redirect
allow_defer
allow_fail
data = ${lookup{$local_part}lsearch{/etc/aliases}}
file_transport = address_file
pipe_transport = address_pipe
retry_use_local_part
# user = exim
local_aliases:
driver = redirect
allow_defer
allow_fail
data = ${lookup{$local_part}lsearch{/etc/localaliases}}
file_transport = address_file
pipe_transport = address_pipe
check_local_user
userforward:
#!!# filter renamed allow_filter
driver = redirect
allow_filter
check_ancestor
check_local_user
domains = ! lsearch;/etc/userdomains
no_expn
file = $home/.forward
file_transport = address_file
pipe_transport = address_pipe
reply_transport = address_reply
no_verify
localuser_spam:
driver = accept
headers_remove="x-spam-exim"
condition = "${perl{checkuserspambox}{$local_part}}"
check_local_user
domains = ! lsearch;/etc/userdomains
transport = local_delivery_spam
localuser:
driver = accept
headers_remove="x-spam-exim"
check_local_user
domains = ! lsearch;/etc/userdomains
transport = local_delivery
Allows the .forward that some people like
Another Box
TRANSPORTS CONFIGURATION (the configurations of the transports used by exim)
Code:
begin transports
Tells Exim that we define the transports here
Code:
remote_smtp:
driver = smtp
This is straight forward, the remote_smtp transport uses the smtp protocol
Code:
local_delivery:
driver = appendfile
delivery_date_add
envelope_to_add
file = "${extract{5}{:}{${lookup{$local_part}lsearch{/etc/passwd}{$value}}}}/mail/inbox"
group = mail
mode = 0660
return_path_add
user = $local_part
Normal email is sent to the mail/inbox folder for the user
Code:
local_delivery_spam:
driver = appendfile
delivery_date_add
envelope_to_add
file = "${extract{5}{:}{${lookup{$local_part}lsearch{/etc/passwd}{$value}}}}/mail/spam"
group = mail
mode = 0660
return_path_add
user = $local_part
This sends email marked as spam to the mail/spam folder
Code:
local_sa_delivery:
driver = pipe
command = /usr/sbin/sendmail -bS
use_bsmtp = true
transport_filter = "/usr/bin/spamc"
user = $local_part
group = mail
log_output = true
current_directory = "/tmp"
home_directory = "/tmp"
return_fail_output = true
return_path_add = false
message_prefix =
message_suffix =
Sends the email through spamassasin
Code:
address_pipe:
driver = pipe
return_output
This pipes email through
Code:
virtual_address_pipe:
driver = pipe
group = "${lookup{$domain}lsearch* {/etc/userdomains}{$value}}"
return_output
user = "${lookup{$domain}lsearch* {/etc/userdomains}{$value}}"
Also just helps with piping the email through to the location request in the valias.
Code:
address_file:
driver = appendfile
delivery_date_add
envelope_to_add
return_path_add
This transport is used for handling deliveries directly to files that are generated by aliasing or forwarding.
Code:
virtual_sa_userdelivery:
driver = pipe
command = /usr/sbin/sendmail -bS
use_bsmtp = true
transport_filter = "/usr/bin/spamc"
user = "${lookup{$domain}lsearch* {/etc/userdomains}{$value}}"
group = mail
log_output = true
current_directory = "/tmp"
home_directory = "/tmp"
return_fail_output = true
return_path_add = false
message_prefix =
message_suffix =
virtual_userdelivery_spam:
driver = appendfile
delivery_date_add
envelope_to_add
file = "${extract{5}{:}{${lookup{${lookup{$domain}lsearch*{/etc/userdomains}{$value}}}lsearch{/etc/passwd}{$value}}}}/mail/${domain}/${local_part}/spam"
group = mail
mode = 0660
quota = "${if exists{${extract{5}{:}{${lookup{${lookup{$domain}lsearch*{/etc/userdomains}{$value}}}lsearch{/etc/passwd}{$value}}}}/etc/${domain}/quota} {${lookup{$local_part}lsearch*{${extract{5}{:}{${lookup{${lookup{$domain}lsearch*{/etc/userdomains}{$value}}}lsearch{/etc/passwd}{$value}}}}/etc/${domain}/quota}{$value}}} {}}"
return_path_add
user = "${lookup{$domain}lsearch* {/etc/userdomains}{$value}}"
virtual_userdelivery:
driver = appendfile
delivery_date_add
envelope_to_add
file = "${extract{5}{:}{${lookup{${lookup{$domain}lsearch*{/etc/userdomains}{$value}}}lsearch{/etc/passwd}{$value}}}}/mail/${domain}/${local_part}/inbox"
group = mail
mode = 0660
quota = "${if exists{${extract{5}{:}{${lookup{${lookup{$domain}lsearch*{/etc/userdomains}{$value}}}lsearch{/etc/passwd}{$value}}}}/etc/${domain}/quota} {${lookup{$local_part}lsearch*{${extract{5}{:}{${lookup{${lookup{$domain}lsearch*{/etc/userdomains}{$value}}}lsearch{/etc/passwd}{$value}}}}/etc/${domain}/quota}{$value}}} {}}"
return_path_add
user = "${lookup{$domain}lsearch* {/etc/userdomains}{$value}}"
This transport is used for handling autoreplies generated by the filtering option of the forwardfile director.
Code:
address_reply:
driver = autoreply
This is for autoreplies
Code:
mailman_virtual_transport:
driver = pipe
command = /usr/local/cpanel/3rdparty/mailman/mail/mailman \
'${if def:local_part_suffix \
{${sg{$local_part_suffix}{-(\\w+)(\\+.*)?}{\$1}}} \
{post}}' \
${lc:$local_part}_${lc:$domain}
current_directory = /usr/local/cpanel/3rdparty/mailman
home_directory = /usr/local/cpanel/3rdparty/mailman
user = mailman
group = mailman
mailman_virtual_transport_nodns:
driver = pipe
command = /usr/local/cpanel/3rdparty/mailman/mail/mailman \
'${if def:local_part_suffix \
{${sg{$local_part_suffix}{-(\\w+)(\\+.*)?}{\$1}}} \
{post}}' \
${lc:$local_part}
current_directory = /usr/local/cpanel/3rdparty/mailman
home_directory = /usr/local/cpanel/3rdparty/mailman
user = mailman
group = mailman
Mailman transports
RETRY CONFIGURATION
Code:
begin retry
Tells Exim that we are starting the retry rules
Code:
* * F,2h,15m; G,16h,1h,1.5; F,4d,8h
This single retry rule applies to all domains and all errors. It specifies retries every 15 minutes for 2 hours, then increasing retry intervals, starting at 1 hour and increasing each time by a factor of 1.5, up to 16 hours, then retries every 8 hours until 4 days have passed since the first failed delivery.
That is your Exim config files and how Exim works on your cPanel box.
MYSQL
MYSQL
Starting MySQL
You have to start the MySQL process before you can create your databases. To configure MySQL to start at boot time, use the chkconfig command:
[root@bigboy tmp]# chkconfig mysqld on
You can start, stop, and restart MySQL after boot time using the service commands.
[root@bigboy tmp]# service mysqld start
[root@bigboy tmp]# service mysqld stop
[root@bigboy tmp]# service mysqld restart
Remember to restart the mysqld process every time you make a change to the configuration file for the changes to take effect on the running process.
You can test whether the mysqld process is running with
[root@bigboy tmp]# pgrep mysqld
You should get a response of plain old process ID numbers.
The /etc/my.cnf File
The /etc/my.cnf file is the main MySQL configuration file. It sets the default MySQL database location and other parameters. The typical home/SOHO user won't need to edit this file at all.
The Location of MySQL Databases
According to the /etc/my.cnf file, MySQL databases are usually located in a subdirectory of the /var/lib/mysql/ directory. If you create a database named test, then the database files will be located in the directory /var/lib/mysql/test.
Creating a MySQL "root" Account
MySQL stores all its username and password data in a special database named mysql. You can add users to this database and specify the databases to which they will have access with the grant command. The MySQL root or superuser account, which is used to create and delete databases, is the exception. You need to use the mysqladmin command to set your root password. Only two steps are necessary for a brand new MySQL installation.
1. Make sure MySQL is started.
2. Use the mysqladmin command to set the MySQL root password. The syntax is as follows:
[root@tmp bigboy]# mysqladmin -u root password new-password
If you want to change your password later, you will probably have to do a root password recovery.
Accessing The MySQL Command Line
MySQL has its own command line interpreter (CLI). You need to know how to access it to do very basic administration.
You can access the MySQL CLI using the mysql command followed by the -u option for the username and -p, which tells MySQL to prompt for a password. Here user root gains access:
[root@bigboy tmp]# mysql -u root -p
Enter password:
Welcome to the MySQL monitor. Commands end with ; or \g.
Your MySQL connection id is 14 to server version: 3.23.58
Type 'help;' or '\h' for help. Type '\c' to clear the buffer.
mysql>
Note: Almost all MySQL CLI commands need to end with a semi-colon. Even the exit command used to get back to the Linux prompt needs one too!
Creating and Deleting MySQL Databases
Many Linux applications that use MySQL databases require you to create the database beforehand using the name of your choice. The procedure is relatively simple: Enter the MySQL CLI, and use the create database command:
mysql> create database salesdata;
Query OK, 1 row affected (0.00 sec)
mysql>
If you make a mistake during the installation process and need to delete the database, use the drop database command. The example deletes the newly created database named salesdata.
mysql> drop database salesdata;
Query OK, 0 rows affected (0.00 sec)
mysql>
Note: Sometimes a dropped database may still appear listed when you use the show databases command explained further below. This may happen even if your root user has been granted full privileges to the database, and it is usually caused by the presence of residual database files in your database directory. In such a case you may have to physically delete the database sub-directory in /var/lib/mysql from the Linux command line. Make sure you stop MySQL before you do this.
[root@bigboy tmp]# service mysqld stop
Granting Privileges to Users
On many occasions you will not only have to create a database, but also have to create a MySQL username and password with privileges to access the database. It is not a good idea to use the root account to do this because of its universal privileges.
MySQL stores all its username and password data in a special database named mysql. You can add users to this database and specify the databases to which they will have access with the grant command, which has the syntax.
sql> grant all privileges on database.* to username@"servername" identified by 'password';
So you can create a user named mysqluser with a password of pinksl1p to have full access to the database named salesdata on the local server (localhost) with the grant
command. If the database application's client resides on another server, then you'll want to replace the localhost address with the actual IP address of that client.
sql> grant all privileges on salesdata.* to mysqluser@"localhost" identified by 'pinksl1p';
The next step is to write the privilege changes to the mysql.sql database using the flush privileges command.
sql> flush privileges;
CREATE DATABASE
• Create a database: (Creates directory /var/lib/mysql/bedrock)
[prompt]$ mysqladmin -h localhost -u root -ppassword create bedrock
(or use SQL command: CREATE DATABASE bedrock;)
• Add tables, data, etc:
Connect to database and issue the following SQL commands:
[prompt]$ mysql -h localhost -u root -ppassword
...
mysql> use bedrock; - Define database to connect to. Refers to directory path: /var/lib/mysql/bedrock
mysql> create table employee (Name char(20),Dept char(20),jobTitle char(20));
mysql> DESCRIBE employee; - View the table just created. Same as "show columns from employee;"
+----------+----------+------+-----+---------+-------+
| Field | Type | Null | Key | Default | Extra |
+----------+----------+------+-----+---------+-------+
| Name | char(20) | YES | | NULL | |
| Dept | char(20) | YES | | NULL | |
| jobTitle | char(20) | YES | | NULL | |
+----------+----------+------+-----+---------+-------+
3 rows in set (0.03 sec)
mysql> show tables;
+-------------------+
| Tables_in_bedrock |
+-------------------+
| employee |
+-------------------+
mysql> INSERT INTO employee VALUES ('Fred Flinstone','Quarry Worker','Rock Digger');
mysql> INSERT INTO employee VALUES ('Wilma Flinstone','Finance','Analyst');
mysql> INSERT into employee values ('Barney Rubble','Sales','Neighbor');
mysql> INSERT INTO employee VALUES ('Betty Rubble','IT','Neighbor');
Note: Data type used was CHAR. Other data types include:
• CHAR(M) : Fixed length string. Always stores M characters whether it is holding 2 or 20 characters. Where M can range 1 to 255 characters.
• VARCHAR(M) : Variable length. Stores only the string. If M is defined to be 200 but the string is 20 characters long, only 20 characters are stored. Slower than CHAR.
• INT : Ranging from -2147483648 to 2147483647 or unsigned 0 to 4294967295
• FLOAT(M,N) : FLOAT(4,2) - Four digits total of which 2 are after the decimal. i.e. 12.34 Values are rounded to fit format if they are too large.
• DATE, TEXT, BLOB, SET, ENUM
• Add a user. Use the MySQL SQL console to enter SQL commands. The command mysql with the correct login/password will connect you to the database. The admin tables are stored in the database "mysql".
• [prompt]$ mysql -h localhost -u root -ppassword
Welcome to the MySQL monitor. Commands end with ; or \g.
Your MySQL connection id is 1 to server version: 3.23.41
Type 'help;' or '\h' for help. Type '\c' to clear the buffer.
mysql> USE mysql;
mysql> SHOW TABLES;
+-----------------+
| Tables_in_mysql |
+-----------------+
| columns_priv |
| db |
| func |
| host |
| tables_priv |
| user |
+-----------------+
mysql> INSERT INTO user (Host, User, Password, Select_priv) VALUES ('', 'Dude1', password('supersecret'), 'Y');
mysql> FLUSH PRIVILEGES; - Required each time one makes a change to the GRANT table
mysql> GRANT ALL PRIVILEGES ON bedrock.* TO Dude1;
mysql> FLUSH PRIVILEGES; - Required each time one makes a change to the GRANT table
mysql> quit
Note:
• There is NO space between the -p and the password! You can omit the password and you will be prompted for it.
• The SQL flush command is equivalent to issuing the command:
[prompt]$ mysqladmin reload
• Test the database:
mysql> SELECT * from employee;
+-----------------+---------------+-------------+
| Name | Dept | jobTitle |
+-----------------+---------------+-------------+
| Fred Flinstone | Quarry Worker | Rock Digger |
| Wilma Flinstone | Finance | Analyst |
| Barney Rubble | Sales | Neighbor |
| Betty Rubble | IT | Neighbor |
+-----------------+---------------+-------------+
1 row in set (0.00 sec)
mysql> SELECT name FROM employee WHERE dept='Sales';
+---------------+
| name |
+---------------+
| Barney Rubble |
+---------------+
1 row in set (0.00 sec)
• Quit from the SQL shell:
[prompt]$ quit
• Shutting down the database:
[prompt]$ mysqladmin -u root -ppassword shutdown - PREFERRED
OR
[prompt]$ /etc/rc.d/init.d/mysqld stop
OR
[prompt]$ service mysqld stop
Viewing Your New MySQL Databases
A number of commands can provide information about your newly created database. Here are some examples:
• Login As The Database User: It is best to do all your database testing as the MySQL user you want the application to eventually use. This will make your testing mimic the actions of the application and results in better testing in a more production-like environment than using the root account.
[root@bigboy tmp]# mysql -u mysqluser -p salesdata
• List all your MySQL databases: The show databases command gives you a list of all your available MySQL databases. In the example, you can see that the salesdata database has been successfully created:
mysql> show databases;
+-----------+
| Database |
+-----------+
| salesdata |
+-----------+
1 row in set (0.00 sec)
mysql>
Listing The Data Tables In Your MySQL Database
The show tables command gives you a list of all the tables in your MySQL database, but you have to use the use command first to tell MySQL to which database it should apply the show tables command.
The example uses the salesdata database; notice that it has a table named test.
mysql> use salesdata;
Reading table information for completion of table and column names
You can turn off this feature to get a quicker startup with -A
Database changed
mysql> show tables;
+---------------------+
| Tables_in_salesdata |
+---------------------+
| test |
+---------------------+
1 row in set (0.00 sec)
mysql>
Viewing Your MySQL Database's Table Structure
The describe command gives you a list of all the data fields used in your database table. In the example, you can see that the table named test in the salesdata database keeps track of four fields: name, description, num, and date_modified.
mysql> describe test;
+---------------+--------------+------+-----+------------+----------------+
| Field | Type | Null | Key | Default | Extra |
+---------------+--------------+------+-----+------------+----------------+
| num | int(11) | | PRI | NULL | auto_increment |
| date_modified | date | | MUL | 0000-00-00 | |
| name | varchar(50) | | MUL | | |
| description | varchar(75) | YES | | NULL | |
+---------------+--------------+------+-----+------------+----------------+
6 rows in set (0.00 sec)
mysql>
Viewing The Contents Of A Table
You can view all the data contained in the table named test by using the select command. In this example you want to see all the data contained in the very first row in the table.
mysql> select * from test limit 1;
With a brand new database this will give a blank listing, but once the application starts and you enter data, you may want to run this command again as a rudimentary database sanity check.
Recovering / Changing Your MySQL Root Password
Sometimes you may have to recover the MySQL root password because it was either forgotten or misplaced. The steps you need are:
1) Stop MySQL
[root@bigboy tmp]# service mysqld stop
Stopping MySQL: [ OK ]
[root@bigboy tmp]#
2) Start MySQL in Safe mode with the mysqld_safe command and tell it not to read the grant tables with all the MySQL database passwords.
[root@bigboy tmp]# mysqld_safe --skip-grant-tables --skip-networking &
[1] 13007
[root@bigboy tmp]# Starting mysqld daemon with databases from /var/lib/mysql
[root@bigboy tmp]#
Note: In Fedora Core 3 and earlier the mysqld_safe command was named safe_mysqld and the general procedure for password recovery was different.
3) MySQL is now running without password protection. You now have to use the familiar mysql -u root command to get the mysql> command prompt. ( -p flag is not required) As expected, you will not be prompted for a password.
[root@bigboy tmp]# mysql -u root
Welcome to the MySQL monitor. Commands end with ; or \g.
Your MySQL connection id is 1 to server version: 4.1.16
Type 'help;' or '\h' for help. Type '\c' to clear the buffer.
mysql>
4) You will now have to use the mysql database which contains the passwords for all the databases on your system and modify the root password. In this case we are setting it to ack33nsaltf1sh.
mysql> use mysql;
Reading table information for completion of table and column names
You can turn off this feature to get a quicker startup with -A
Database changed
mysql> UPDATE user SET Password=PASSWORD("ack33nsaltf1sh") WHERE User="root";
Query OK, 1 row affected (0.00 sec)
Rows matched: 2 Changed: 1 Warnings: 0
mysql>
5) Exit MySQL and restart the mysqld daemon.
mysql> exit
Bye
[root@bigboy tmp]# service mysqld restart
STOPPING server from pid file /var/run/mysqld/mysqld.pid
051224 17:24:56 mysqld ended
Stopping MySQL: [ OK ]
Starting MySQL: [ OK ]
[1]+ Done mysqld_safe --skip-grant-tables --skip-networking
[root@bigboy tmp]#
The MySQL root user will now be able to manage MySQL using this new password.
MySQL Database Backup
The syntax for backing up a MySQL database is as follows:
mysqldump --add-drop-table -u [username] -p[password] [database] > [backup_file]
In the previous section, you gave user mysqluser full access to the salesdata database when mysqluser used the password pinksl1p. You can now back up this database to a single file called /tmp/salesdata-backup.sql with the command
[root@bigboy tmp]# mysqldump --add-drop-table -u mysqluser \
-ppinksl1p salesdata > /tmp/salesdata-backup.sql
Make sure there are no spaces between the -p switch and the password or else you may get syntax errors.
Note: Always backup the database named mysql too, because it contains all the database user access information.
Eg:
mysqldump --opt server4sale > /tmp/server4sale.sql
MySQL Database Restoration
The syntax for restoring a MySQL database is:
mysql -u [username] -p[password] [database] < [backup_file]
So, using the previous example, you can restore the contents of the database with
[root@bigboy tmp]# mysql -u mysqluser -ppinksl1p salesdata \
< /tmp/salesdata-backup.sql
Note: You may have to restore the database named mysql also, because it contains all the database user access information.
Eg:
mysql -u root -p server4sale < /tmp/server4sale.sql
MySQL Table Backup and Restoration
Sometimes you may want to backup only one or more tables from a database. There are some practical reasons for wanting to do this. You may have a message board / forums application that uses MySQL to store its data and you want to create a brand new forum with the same users as the old one so that the users don't have to register all over again.
The MySQL SELECT statement can be used to export the data to a backup file and the LOAD command can be used to import the data back into the new database used by the new forum. In this example the data in the phpbb_users and phpbb_themes tables of the forums-db-old database are exported to files named /tmp/forums-db-users.sql and /tmp/forums-db-themes.sql respectively. The data is then imported into tables of the same name in the forums-db-new database.
mysql> use forums-db-old;
Reading table information for completion of table and column names
You can turn off this feature to get a quicker startup with -A
Database changed
mysql> SELECT * INTO OUTFILE '/tmp/forums-db-users.sql' FROM phpbb_users;
Query OK, 1042 rows affected (0.03 sec)
mysql> SELECT * INTO OUTFILE '/tmp/forums-db-themes.sql' FROM phpbb_themes;
Query OK, 1038 rows affected (0.03 sec)
mysql> use forums-db-new;
Database changed
mysql> load data infile '/tmp/forums-db-users.sql' replace into table forums-db.phpbb_users ;
Query OK, 1042 rows affected (0.06 sec)
Records: 1042 Deleted: 0 Skipped: 0 Warnings: 0
mysql> load data infile '/tmp/forums-db-themes.sql' replace into table forums-db.phpbb_themes ;
Query OK, 1038 rows affected (0.04 sec)
Records: 1038 Deleted: 0 Skipped: 0 Warnings: 0
mysql>
As you can see, the syntax is fairly easy to understand. The REPLACE directive will overwrite any previously existing records with the same unique, or primary, key in the source and destination tables. The IGNORE directive will only insert records where the primary keys are different.
Starting MySQL
You have to start the MySQL process before you can create your databases. To configure MySQL to start at boot time, use the chkconfig command:
[root@bigboy tmp]# chkconfig mysqld on
You can start, stop, and restart MySQL after boot time using the service commands.
[root@bigboy tmp]# service mysqld start
[root@bigboy tmp]# service mysqld stop
[root@bigboy tmp]# service mysqld restart
Remember to restart the mysqld process every time you make a change to the configuration file for the changes to take effect on the running process.
You can test whether the mysqld process is running with
[root@bigboy tmp]# pgrep mysqld
You should get a response of plain old process ID numbers.
The /etc/my.cnf File
The /etc/my.cnf file is the main MySQL configuration file. It sets the default MySQL database location and other parameters. The typical home/SOHO user won't need to edit this file at all.
The Location of MySQL Databases
According to the /etc/my.cnf file, MySQL databases are usually located in a subdirectory of the /var/lib/mysql/ directory. If you create a database named test, then the database files will be located in the directory /var/lib/mysql/test.
Creating a MySQL "root" Account
MySQL stores all its username and password data in a special database named mysql. You can add users to this database and specify the databases to which they will have access with the grant command. The MySQL root or superuser account, which is used to create and delete databases, is the exception. You need to use the mysqladmin command to set your root password. Only two steps are necessary for a brand new MySQL installation.
1. Make sure MySQL is started.
2. Use the mysqladmin command to set the MySQL root password. The syntax is as follows:
[root@tmp bigboy]# mysqladmin -u root password new-password
If you want to change your password later, you will probably have to do a root password recovery.
Accessing The MySQL Command Line
MySQL has its own command line interpreter (CLI). You need to know how to access it to do very basic administration.
You can access the MySQL CLI using the mysql command followed by the -u option for the username and -p, which tells MySQL to prompt for a password. Here user root gains access:
[root@bigboy tmp]# mysql -u root -p
Enter password:
Welcome to the MySQL monitor. Commands end with ; or \g.
Your MySQL connection id is 14 to server version: 3.23.58
Type 'help;' or '\h' for help. Type '\c' to clear the buffer.
mysql>
Note: Almost all MySQL CLI commands need to end with a semi-colon. Even the exit command used to get back to the Linux prompt needs one too!
Creating and Deleting MySQL Databases
Many Linux applications that use MySQL databases require you to create the database beforehand using the name of your choice. The procedure is relatively simple: Enter the MySQL CLI, and use the create database command:
mysql> create database salesdata;
Query OK, 1 row affected (0.00 sec)
mysql>
If you make a mistake during the installation process and need to delete the database, use the drop database command. The example deletes the newly created database named salesdata.
mysql> drop database salesdata;
Query OK, 0 rows affected (0.00 sec)
mysql>
Note: Sometimes a dropped database may still appear listed when you use the show databases command explained further below. This may happen even if your root user has been granted full privileges to the database, and it is usually caused by the presence of residual database files in your database directory. In such a case you may have to physically delete the database sub-directory in /var/lib/mysql from the Linux command line. Make sure you stop MySQL before you do this.
[root@bigboy tmp]# service mysqld stop
Granting Privileges to Users
On many occasions you will not only have to create a database, but also have to create a MySQL username and password with privileges to access the database. It is not a good idea to use the root account to do this because of its universal privileges.
MySQL stores all its username and password data in a special database named mysql. You can add users to this database and specify the databases to which they will have access with the grant command, which has the syntax.
sql> grant all privileges on database.* to username@"servername" identified by 'password';
So you can create a user named mysqluser with a password of pinksl1p to have full access to the database named salesdata on the local server (localhost) with the grant
command. If the database application's client resides on another server, then you'll want to replace the localhost address with the actual IP address of that client.
sql> grant all privileges on salesdata.* to mysqluser@"localhost" identified by 'pinksl1p';
The next step is to write the privilege changes to the mysql.sql database using the flush privileges command.
sql> flush privileges;
CREATE DATABASE
• Create a database: (Creates directory /var/lib/mysql/bedrock)
[prompt]$ mysqladmin -h localhost -u root -ppassword create bedrock
(or use SQL command: CREATE DATABASE bedrock;)
• Add tables, data, etc:
Connect to database and issue the following SQL commands:
[prompt]$ mysql -h localhost -u root -ppassword
...
mysql> use bedrock; - Define database to connect to. Refers to directory path: /var/lib/mysql/bedrock
mysql> create table employee (Name char(20),Dept char(20),jobTitle char(20));
mysql> DESCRIBE employee; - View the table just created. Same as "show columns from employee;"
+----------+----------+------+-----+---------+-------+
| Field | Type | Null | Key | Default | Extra |
+----------+----------+------+-----+---------+-------+
| Name | char(20) | YES | | NULL | |
| Dept | char(20) | YES | | NULL | |
| jobTitle | char(20) | YES | | NULL | |
+----------+----------+------+-----+---------+-------+
3 rows in set (0.03 sec)
mysql> show tables;
+-------------------+
| Tables_in_bedrock |
+-------------------+
| employee |
+-------------------+
mysql> INSERT INTO employee VALUES ('Fred Flinstone','Quarry Worker','Rock Digger');
mysql> INSERT INTO employee VALUES ('Wilma Flinstone','Finance','Analyst');
mysql> INSERT into employee values ('Barney Rubble','Sales','Neighbor');
mysql> INSERT INTO employee VALUES ('Betty Rubble','IT','Neighbor');
Note: Data type used was CHAR. Other data types include:
• CHAR(M) : Fixed length string. Always stores M characters whether it is holding 2 or 20 characters. Where M can range 1 to 255 characters.
• VARCHAR(M) : Variable length. Stores only the string. If M is defined to be 200 but the string is 20 characters long, only 20 characters are stored. Slower than CHAR.
• INT : Ranging from -2147483648 to 2147483647 or unsigned 0 to 4294967295
• FLOAT(M,N) : FLOAT(4,2) - Four digits total of which 2 are after the decimal. i.e. 12.34 Values are rounded to fit format if they are too large.
• DATE, TEXT, BLOB, SET, ENUM
• Add a user. Use the MySQL SQL console to enter SQL commands. The command mysql with the correct login/password will connect you to the database. The admin tables are stored in the database "mysql".
• [prompt]$ mysql -h localhost -u root -ppassword
Welcome to the MySQL monitor. Commands end with ; or \g.
Your MySQL connection id is 1 to server version: 3.23.41
Type 'help;' or '\h' for help. Type '\c' to clear the buffer.
mysql> USE mysql;
mysql> SHOW TABLES;
+-----------------+
| Tables_in_mysql |
+-----------------+
| columns_priv |
| db |
| func |
| host |
| tables_priv |
| user |
+-----------------+
mysql> INSERT INTO user (Host, User, Password, Select_priv) VALUES ('', 'Dude1', password('supersecret'), 'Y');
mysql> FLUSH PRIVILEGES; - Required each time one makes a change to the GRANT table
mysql> GRANT ALL PRIVILEGES ON bedrock.* TO Dude1;
mysql> FLUSH PRIVILEGES; - Required each time one makes a change to the GRANT table
mysql> quit
Note:
• There is NO space between the -p and the password! You can omit the password and you will be prompted for it.
• The SQL flush command is equivalent to issuing the command:
[prompt]$ mysqladmin reload
• Test the database:
mysql> SELECT * from employee;
+-----------------+---------------+-------------+
| Name | Dept | jobTitle |
+-----------------+---------------+-------------+
| Fred Flinstone | Quarry Worker | Rock Digger |
| Wilma Flinstone | Finance | Analyst |
| Barney Rubble | Sales | Neighbor |
| Betty Rubble | IT | Neighbor |
+-----------------+---------------+-------------+
1 row in set (0.00 sec)
mysql> SELECT name FROM employee WHERE dept='Sales';
+---------------+
| name |
+---------------+
| Barney Rubble |
+---------------+
1 row in set (0.00 sec)
• Quit from the SQL shell:
[prompt]$ quit
• Shutting down the database:
[prompt]$ mysqladmin -u root -ppassword shutdown - PREFERRED
OR
[prompt]$ /etc/rc.d/init.d/mysqld stop
OR
[prompt]$ service mysqld stop
Viewing Your New MySQL Databases
A number of commands can provide information about your newly created database. Here are some examples:
• Login As The Database User: It is best to do all your database testing as the MySQL user you want the application to eventually use. This will make your testing mimic the actions of the application and results in better testing in a more production-like environment than using the root account.
[root@bigboy tmp]# mysql -u mysqluser -p salesdata
• List all your MySQL databases: The show databases command gives you a list of all your available MySQL databases. In the example, you can see that the salesdata database has been successfully created:
mysql> show databases;
+-----------+
| Database |
+-----------+
| salesdata |
+-----------+
1 row in set (0.00 sec)
mysql>
Listing The Data Tables In Your MySQL Database
The show tables command gives you a list of all the tables in your MySQL database, but you have to use the use command first to tell MySQL to which database it should apply the show tables command.
The example uses the salesdata database; notice that it has a table named test.
mysql> use salesdata;
Reading table information for completion of table and column names
You can turn off this feature to get a quicker startup with -A
Database changed
mysql> show tables;
+---------------------+
| Tables_in_salesdata |
+---------------------+
| test |
+---------------------+
1 row in set (0.00 sec)
mysql>
Viewing Your MySQL Database's Table Structure
The describe command gives you a list of all the data fields used in your database table. In the example, you can see that the table named test in the salesdata database keeps track of four fields: name, description, num, and date_modified.
mysql> describe test;
+---------------+--------------+------+-----+------------+----------------+
| Field | Type | Null | Key | Default | Extra |
+---------------+--------------+------+-----+------------+----------------+
| num | int(11) | | PRI | NULL | auto_increment |
| date_modified | date | | MUL | 0000-00-00 | |
| name | varchar(50) | | MUL | | |
| description | varchar(75) | YES | | NULL | |
+---------------+--------------+------+-----+------------+----------------+
6 rows in set (0.00 sec)
mysql>
Viewing The Contents Of A Table
You can view all the data contained in the table named test by using the select command. In this example you want to see all the data contained in the very first row in the table.
mysql> select * from test limit 1;
With a brand new database this will give a blank listing, but once the application starts and you enter data, you may want to run this command again as a rudimentary database sanity check.
Recovering / Changing Your MySQL Root Password
Sometimes you may have to recover the MySQL root password because it was either forgotten or misplaced. The steps you need are:
1) Stop MySQL
[root@bigboy tmp]# service mysqld stop
Stopping MySQL: [ OK ]
[root@bigboy tmp]#
2) Start MySQL in Safe mode with the mysqld_safe command and tell it not to read the grant tables with all the MySQL database passwords.
[root@bigboy tmp]# mysqld_safe --skip-grant-tables --skip-networking &
[1] 13007
[root@bigboy tmp]# Starting mysqld daemon with databases from /var/lib/mysql
[root@bigboy tmp]#
Note: In Fedora Core 3 and earlier the mysqld_safe command was named safe_mysqld and the general procedure for password recovery was different.
3) MySQL is now running without password protection. You now have to use the familiar mysql -u root command to get the mysql> command prompt. ( -p flag is not required) As expected, you will not be prompted for a password.
[root@bigboy tmp]# mysql -u root
Welcome to the MySQL monitor. Commands end with ; or \g.
Your MySQL connection id is 1 to server version: 4.1.16
Type 'help;' or '\h' for help. Type '\c' to clear the buffer.
mysql>
4) You will now have to use the mysql database which contains the passwords for all the databases on your system and modify the root password. In this case we are setting it to ack33nsaltf1sh.
mysql> use mysql;
Reading table information for completion of table and column names
You can turn off this feature to get a quicker startup with -A
Database changed
mysql> UPDATE user SET Password=PASSWORD("ack33nsaltf1sh") WHERE User="root";
Query OK, 1 row affected (0.00 sec)
Rows matched: 2 Changed: 1 Warnings: 0
mysql>
5) Exit MySQL and restart the mysqld daemon.
mysql> exit
Bye
[root@bigboy tmp]# service mysqld restart
STOPPING server from pid file /var/run/mysqld/mysqld.pid
051224 17:24:56 mysqld ended
Stopping MySQL: [ OK ]
Starting MySQL: [ OK ]
[1]+ Done mysqld_safe --skip-grant-tables --skip-networking
[root@bigboy tmp]#
The MySQL root user will now be able to manage MySQL using this new password.
MySQL Database Backup
The syntax for backing up a MySQL database is as follows:
mysqldump --add-drop-table -u [username] -p[password] [database] > [backup_file]
In the previous section, you gave user mysqluser full access to the salesdata database when mysqluser used the password pinksl1p. You can now back up this database to a single file called /tmp/salesdata-backup.sql with the command
[root@bigboy tmp]# mysqldump --add-drop-table -u mysqluser \
-ppinksl1p salesdata > /tmp/salesdata-backup.sql
Make sure there are no spaces between the -p switch and the password or else you may get syntax errors.
Note: Always backup the database named mysql too, because it contains all the database user access information.
Eg:
mysqldump --opt server4sale > /tmp/server4sale.sql
MySQL Database Restoration
The syntax for restoring a MySQL database is:
mysql -u [username] -p[password] [database] < [backup_file]
So, using the previous example, you can restore the contents of the database with
[root@bigboy tmp]# mysql -u mysqluser -ppinksl1p salesdata \
< /tmp/salesdata-backup.sql
Note: You may have to restore the database named mysql also, because it contains all the database user access information.
Eg:
mysql -u root -p server4sale < /tmp/server4sale.sql
MySQL Table Backup and Restoration
Sometimes you may want to backup only one or more tables from a database. There are some practical reasons for wanting to do this. You may have a message board / forums application that uses MySQL to store its data and you want to create a brand new forum with the same users as the old one so that the users don't have to register all over again.
The MySQL SELECT statement can be used to export the data to a backup file and the LOAD command can be used to import the data back into the new database used by the new forum. In this example the data in the phpbb_users and phpbb_themes tables of the forums-db-old database are exported to files named /tmp/forums-db-users.sql and /tmp/forums-db-themes.sql respectively. The data is then imported into tables of the same name in the forums-db-new database.
mysql> use forums-db-old;
Reading table information for completion of table and column names
You can turn off this feature to get a quicker startup with -A
Database changed
mysql> SELECT * INTO OUTFILE '/tmp/forums-db-users.sql' FROM phpbb_users;
Query OK, 1042 rows affected (0.03 sec)
mysql> SELECT * INTO OUTFILE '/tmp/forums-db-themes.sql' FROM phpbb_themes;
Query OK, 1038 rows affected (0.03 sec)
mysql> use forums-db-new;
Database changed
mysql> load data infile '/tmp/forums-db-users.sql' replace into table forums-db.phpbb_users ;
Query OK, 1042 rows affected (0.06 sec)
Records: 1042 Deleted: 0 Skipped: 0 Warnings: 0
mysql> load data infile '/tmp/forums-db-themes.sql' replace into table forums-db.phpbb_themes ;
Query OK, 1038 rows affected (0.04 sec)
Records: 1038 Deleted: 0 Skipped: 0 Warnings: 0
mysql>
As you can see, the syntax is fairly easy to understand. The REPLACE directive will overwrite any previously existing records with the same unique, or primary, key in the source and destination tables. The IGNORE directive will only insert records where the primary keys are different.
Apache 2 and PHP
Apache 2 and PHP Installation
The following notes are how I got Apache 2 and PHP 5 (or PHP 4) working together on Linux. These instructions also apply, mostly, for any UNIX-like system, especially other Linux distributions. If you have a recent Linux distribution (say since 2002), you already have Apache 2 and PHP, so you don't need to do this unless you want the latest Apache 2 or PHP release or need to customize the Apache or PHP software. Don't forget to remove (or at least disable) the Apache rpm package if you install your own custom Apache.
Apache 2 Version Tip: Beginning with Apache 2.0.42 the API will be kept stable (yeah!). That means you will NOT have to recompile modules (and possibly upgrade/fix source) every time you install a new Apache release. This assumes you stay in the same Apache release series. For example, upgrading from 2.2.0 to 2.2.2 should work. This will not apply to upgrading to the next series (e.g., "development" 2.3.x or "stable" 2.4.x).
I was only able to get PHP working with Apache 2 as a *.so DSO ("dynamic shared object"), as opposed to compiled into Apache (with a static *.a library). I think DSO is the only way PHP is supported now on Apache 2.
I first used httpd-2.0.43 and php-4.3.0 with RedHat 7.3. I am now using httpd-2.2.3 and php-5.2.0 with SUSE 10.1.
Note: If you have problems with PHP and think it's a recent bug, you may want to consider using the latest http://snaps.php.net/ snapshot. Beware that snapshots frequently have regression and are not for production use. Usually problems are because of mis-configuration, not bugs, so snapshots will probably hurt more than help.
1. Download/unpack Apache2 source from the Apache httpd server website, http://httpd.apache.org/
2. In the Apache 2 source directory, create a Makefile by typing:
./configure --prefix=/usr/local/apache \
--enable-so \
--enable-cgi \
--enable-info \
--enable-rewrite \
--enable-speling \
--enable-usertrack \
--enable-deflate \
--enable-ssl \
--enable-mime-magic
You only need the enable-so line above. For information on other options, type ./configure --help and see "Compiling and Installing" in the Apache 2 Documentation, http://httpd.apache.org/
3. Make Apache from the just-created Makefile:
make
4. If make is successful, install Apache as root:
make install
5. Download/unpack PHP source from the PHP website, http://www.php.net/
Pick the latest from the 4.x series or 5.x series.
6. In the PHP source directory, create a Makefile by typing:
./configure \
--with-apxs2=/usr/local/apache/bin/apxs \
--with-mysql \
--prefix=/usr/local/apache/php \
--with-config-file-path=/usr/local/apache/php \
--enable-force-cgi-redirect \
--disable-cgi \
--with-zlib \
--with-gettext \
--with-gdbm
You only need the --with-apxs2, and prefix lines. --with-mysql adds MySql, --with-config-file moves the php.ini file location, disable-cgi disables the CGI version, which is not needed if you use Apache modules. It also enables and installs the command line interface (CLI) version. --with-zlib allows use of gzip-type compression, --with-gettext is for internationalization, and --with-gdbm allows access to GDBM databases. For more information, type ./configure --help and see the "Installation" chapter in the PHP Manual, http://ww.php.net/docs.php
7. Make PHP from the just-created Makefile:
make
8. If make is successful, type this as root to install PHP:
make install
If you are not root (I do not perform makes while root, for security and safety reasons), become root and type the following:
make install-su
9. If file /usr/local/apache/modules/libphp5.so does not exist or is an older version, type this (change this to libphp4.so for PHP 4):
cp -p .libs/libphp5.so /usr/local/apache/modules
10. Install the php.ini file:
cp -p php.ini-recommended /usr/local/apache/php/php.ini
11. Add these directives are in /usr/local/apache/conf/httpd.conf (if already there, verify they are correct):
# Make sure there's only **1** line for each of these 2 directives:
# Use for PHP 4.x:
#LoadModule php4_module modules/libphp4.so
#AddHandler php-script php
# Use for PHP 5.x:
LoadModule php5_module modules/libphp5.so
AddHandler php5-script php
# Add index.php to your DirectoryIndex line:
DirectoryIndex index.html index.php
AddType text/html php
# PHP Syntax Coloring
# (optional but useful for reading PHP source for debugging):
AddType application/x-httpd-php-source phps
Note 1: The php documentation recommends AddType application/x-httpd-php php instead of the above. However, it causes problems with the "MultiViews" feature of HTTP. That is, if the .php extension is left off a URL, and with certain browser Accept headers, Apache will not know .php (application/x-httpd-php) is HTML and will return a 406 Not Acceptable error. Using the AddType and AddHandler as shown above fixes this problem. For details see Mark Tranchant's webpage, "Using Apache's MultiViews with PHP whilst avoid 406 errors," and PHP bug 28023.
Note 2: PHP Syntax coloring isn't required, but it's very nice for looking at your php source while debugging. Here's an example.
Note 3: just for completeness I'll mention that you should be able to use SetOutputFilter / SetInputFilter instead of AddType, but you can't use both. However, SetOutputFilter / SetInputFilter no longer works for me. It used to work with an earlier PHP 4.x or Apache 2 version, but not with Apache 2.0.47/PHP 4.3.3. I understand this (PHP as an Apache 2 filter) is experimental, so I don't use it anymore:
SetOutputFilter PHP
SetInputFilter PHP
12. You're now ready to try it out. Start Apache (httpd) as root:
/usr/local/apache/bin/apachectl start
13. Perform these sanity checks to verify your install went OK:
$ /usr/local/apache/bin/httpd -t
Syntax OK
$ /usr/local/apache/bin/httpd -v
Server version: Apache/2.2.2
Server built: May 29 2006 12:40:55
$ /usr/local/apache/bin/httpd -V
Server version: Apache/2.2.2
Server built: May 29 2006 12:40:55
Server's Module Magic Number: 20051115:2
Server loaded: APR 1.2.7, APR-Util 1.2.7
Compiled using: APR 1.2.7, APR-Util 1.2.7
Architecture: 32-bit
Server MPM: Prefork
threaded: no
forked: yes (variable process count)
Server compiled with....
-D APACHE_MPM_DIR="server/mpm/prefork"
-D APR_HAS_SENDFILE
-D APR_HAS_MMAP
-D APR_HAVE_IPV6 (IPv4-mapped addresses enabled)
-D APR_USE_SYSVSEM_SERIALIZE
-D APR_USE_PTHREAD_SERIALIZE
-D SINGLE_LISTEN_UNSERIALIZED_ACCEPT
-D APR_HAS_OTHER_CHILD
-D AP_HAVE_RELIABLE_PIPED_LOGS
-D DYNAMIC_MODULE_LIMIT=128
-D HTTPD_ROOT="/usr/local/apache"
-D SUEXEC_BIN="/usr/local/apache/bin/suexec"
-D DEFAULT_PIDLOG="logs/httpd.pid"
-D DEFAULT_SCOREBOARD="logs/apache_runtime_status"
-D DEFAULT_LOCKFILE="logs/accept.lock"
-D DEFAULT_ERRORLOG="logs/error_log"
-D AP_TYPES_CONFIG_FILE="conf/mime.types"
-D SERVER_CONFIG_FILE="conf/httpd.conf"
$ /usr/local/apache/bin/httpd -S
VirtualHost configuration:
. . .
$ /usr/local/apache/bin/httpd -l
Compiled in modules:
core.c
. . .
mod_so.c
$ /usr/local/apache/bin/httpd -M
Loaded Modules:
. . .
php5_module (shared)
Syntax OK
(the above works for Apache 2.2.x and higher only)
$ ps -ef |grep httpd
root 24069 1 0 09:17 ? 00:00:08 /usr/local/apache/bin/httpd -k s
apache 29917 24069 0 15:30 ? 00:00:00 /usr/local/apache/bin/httpd -k s
. . .
Note: on BSD-based UNIX systems, you need to use "ps -aux" instead of "ps -ef".
14. Access your webserver with telnet. Type HEAD / HTTP/1.0 followed by a blank line:
$ telnet localhost 80
Trying 127.0.0.1...
Connected to localhost (127.0.0.1).
Escape character is '^]'.
HEAD / HTTP/1.0
HTTP/1.1 200 OK
Date: Mon, 29 May 2006 23:28:18 GMT
Server: Apache/2.2.2 (Unix) mod_ssl/2.2.2 OpenSSL/0.9.7a PHP/5.1.4
X-Powered-By: PHP/5.1.4
Last-Modified: Wed, 15 Mar 2006 06:53:17 GMT
Vary: Accept-Encoding
Connection: close
Content-Type: text/html; charset=ISO-8859-1
Content-Language: en
Connection closed by foreign host.
15. Access your webserver with your favorite browser. The following is a good test page to use for PHP. You only need the one line in bold is needed to display PHP configuration information. Name the file anything you want, but it must end with .php, such as phpinfo.php, and move the file to your web server content directory (for me /usr/local/apache/htdocs), with read permission set:
Tips and Notes
• Disabling Existing Apache Software
If Apache is already installed with your Linux distribution you have to disable it before installing your own custom version. To do this, type the following under RedHat Linux:
/sbin/chkconfig --del httpd
/etc/init.d/httpd stop
You may also remove the httpd and php rpm packages, and dependent packages, if you wish.
• Automatic Startup of Apache
To start Apache automatically, follow these steps:
● Copy Apache startup file to the startup directory:
cp /usr/local/apache/bin/apachectl /etc/init.d/
● Edit /etc/init.d/apachectl by inserting these 2 lines in bold:
#!/bin/sh
#
# chkconfig: - 85 15
# description: Apache is a Web server used to serve HTML files and CGI.
#
# Copyright 2000-2005 The Apache Software Foundation or its licensors, as
# applicable.
. . .
# Apache control script designed to allow an easy command line interface
# to controlling Apache. Written by Marc Slemko, 1997/08/23
. . .
● Enable httpd to startup automatically:
/sbin/chkconfig --add apachectl
/sbin/chkconfig --level 2 apachectl on
This works for RedHat Linux, Mandrake, SuSE, and Sun Java Desktop System. The equivalent of chkconfig for Debian is:
update-rc.d apachectl defaults
For other distributions, make appropriate softlinks to directories /etc/rc*.d/
● Reboot Linux to verify Apache starts
[Thanks to Beriah Dutcher and others]
• Make doesn't make dynamic library libphp5.so, just static library libphp5.a
First make sure libphp5.so isn't there. It's usually in the libs or .libs subdirectory. Run find from the PHP source directory root: find . -name libphp5.so If that doesn't work, make sure you use the --with-apxs2 option with the correct directory in ./configure. For example:
./configure --with-mysql --with-apxs2=/path/to/apache/apxs
When re-running ./configure always start with a fresh, clean source directory (no object).
(For PHP 4, substitute libphp4.so and libphp4.a for libphp5.so and libphp5.a, respectively.)
• PHP 5: My PHP scripts that worked with PHP 4 now ignores input variables
If you have old PHP 4 scripts you don't want to convert to PHP 5, change this line in your php.ini file:
register_long_arrays = On
For the long term, you may want to upgrade your PHP software or convert the PHP source code to use new-style variables. E.g., use $_REQUEST[i] instead of $i
• PHP 5.2.0 Upgrade from 5.1.x (November 2006)
I had one problem with upgrading from PHP 5.1.x to 5.2.0. The Phorum 3 software I use has a function date_format that now conflicts with a new PHP 5.2 internal function, also called date_format. This new function is used for PHP 5.2.x's new "datetime support". I fixed this by renaming Phorum's date_format function to not conflict with PHP's function. See also PHP 5.2 Update Info for more PHP 5.2 upgrade information.
• PHP 5.x: you may need to upgrade libxml2 to use XML
To use PHP 5 and XML (--with-xml), you need the libxml2 library version 2.6.0 or greater. You may need to upgrade if you have an older Linux distribution. Check with rpm -q libxml2
If it's not installed, check your installation disks for the libxml2 package. If it's too old, you can get the library from XMLSoft.org. You will probably need to compile and install it yourself.
• PHP 4.3.5 / 4.3.6 / 5.0RC1 PCRE Regression (March 2004)
If you use PHP 4.3.5, 4.3.6, or PHP 5.0 RC1 with PCRE (Perl Regular Expressions, the default), you get a core dump when you try and restart (SIGHUP) Apache (at least with Apache 2.0.49). The cause is from adding a new version of PCRE, 4.5, in PHP. This is fixed in PHP 4.3.7 and 5.0.
The symptom are these messages in error_log (which is also a symptom for lots of other memory-type bugs in Apache):
[Wed Mar 31 17:14:43 2004] [notice] SIGHUP received. Attempting to restart
[Wed Mar 31 17:14:43 2004] [notice] seg fault or similar nasty error detected in the parent process
[Wed Mar 31 17:14:48 2004] [warn] pid file /var/run/httpd.pid overwritten -- Unclean shutdown of previous Apache run?
To fix, upgrade to PHP 4.3.7 or higher, which fixes it (PHP 5.0 or higher also fixes the bug). A workaround is to stop/start Apache, instead of restarting Apache.
• ServerRoot Installation Directory
By default, Apache is installed in ServerRoot, as specified in httpd.conf or the httpd -d option:
ServerRoot "/usr/local/apache"
Some packagers and distributions move the location to /usr/local/httpd or /usr/local/apache2 so check that Apache is installed where you say it is, and check that you don't have two separate Apache installations by mistake.
• PHP-required Apache files must be present
PHP expects certain files to be present. They may be missing for some Linux distributions, which tend to separate files among the /var, /etc, and other directories. The apxs program expects to find httpd under the bin directory where apache is installed (by default /usr/local/apache/bin). PHP expects to find the os.h and unixd.h header files under the include directory where apache is installed (by default /usr/local/apache/include).
• MySQL
Use the --with-mysql configure option to build PHP with PHP's version of MySql. If you want the system's MySQL, or are also using MySql with another apache module (e.g., mod_perl), use this: with-mysql=/usr
• Custom OpenSSL and SSL
When using "--enable-ssl" (i.e, build with OpenSSL), the configure script will try to guess the location of OpenSSL. If you're using your own custom SSL, you must specify the location. For example, "--with-ssl=/usr/local/ssl" (of course, you can use the SSL and PHP that comes with Redhat 9 and other distributions).
Please note that mod_ssl is only for Apache 1.x. Apache 2.x does not use mod_ssl—SSL is built into Apache 2.x.
Jason reports that with RedHat 9 on his system, the make failed at mod_ssl. It needed a missing krb5.h header file. The fix is to Add an environment variable, $CFLAGS, before making PHP. That is:
"-I/usr/kerberos/include/ -L/usr/kerberos/lib"
• gdbm
If you get this error when using "--with-gdbm":
configure: error: DBA: Could not find necessary header file(s).
then you are probably missing the gdbm-devel rpm package.
• Apache 1.x and Apache 2 Configuration Directives
The "AddModule," "AgentLog," "ReferrerLog," "RefererIgnore," "BindAddress," "ClearModuleList," "Port," "FancyIndexing," "ServerType," "AccessConfig," and "ResourceConfig" directives in Apache 1.x no longer exist in Apache 2. Also, "HAVE_*" httpd.conf definitions no longer exist. Comment them out of your httpd.conf for Apache 2. Most of these directives have been replaced by other directive(s).
• Error after glibc library upgrade
Apache should still work after upgrading glibc with security and bug fixes for the same release. If you use another architecture (say from or to i386/i686), you need to recompile. One symptom is an error like this in your Apache error_log:
[error] (38)Function not implemented:
• make clean
Kåre Olaussen adds this note: If you're compiling php4 for both Apache 1.3 and Apache 2, use a separate php source directory for each version of Apache. "The module wouldn't load into Apache 2, because I had forgotten to run make clean in between." [Ed. note: I recommend untaring the source from scratch in another directory, that way you are sure no old state information is left behind from previous makes. Do this for any configuration or version changes.]
• Enabling register_globals for Apache 2
From Duke: For those of you that are desperately trying to enable register_globals on a per-directory basis using Apache 2 with PHP 4 [or PHP 5] using .htaccess files, the syntax is different from Apache 1.x. In the directory you want to enable register_globals (or pretty much any other on/off setting), add an .htaccess file containing:
php_value register_globals 1
Note that php_flag does not work with Apache 2. Neither does on / off — you must use 0 (off) or 1 (on).
[Editor's note: To enable globally, add register_globals=off to your php.ini file.]
To enable register_globals on a per-directory basis, you can add to a stanza or create a .htaccess configuration file in the directory. To use .htaccess, you must first enable it by adding AllowOverride All (or some combination of AllowOverride options within the correct stanza. Also newer PHP software, such as SquirrelMail, no longer require register_globals to be enabled. Keeping register_globals disalbed makes the system much more secure (as it's harder to set internal php variables through submitting html forms). ]
• Multi-Processing Module (MPM): don't use multi-threading MPMs
MPMs is how Apache 2 handles multiple web server requests. That is, with multiple processes or multiple threads or some combination. For now, on Linux (and even UNIX) you should only use the (default) prefork module with PHP. This is specified at compile time. Other MPM modules (any involving threads) break PHP. This is partly because PHP uses a great number of external libraries, and many or most of them are not thread-safe or thread-aware. In any case, Linux 2.4 doesn't handle threads efficiently yet—multiple processes are better (this changes with Linux 2.6, or RedHat 9 with 2.6 threads backported to Linux 2.4).
• API module structure `php5_module' in file . . . is garbled
If you get a message like this:
httpd: Syntax error on line 195 of /usr/local/apache/conf/httpd.conf: API module
structure `php5_module' in file /usr/local/apache/modules/libphp5.so is garbled
- perhaps this is not an Apache module DSO
it's because your version of PHP wasn't built for the version of Apache you are using (For PHP 4, substitute php4_module . . . libphp4.so). Recompile PHP in a "fresh" source directory (where you haven't built PHP before) and make sure --prefix, --with-config-file and --with-apxs2 points to the correct Apache directories you are using and the libphp4.so file has been copied to your /usr/local/apache/modules directory.
• PATH_INFO
Dan Fitzpatrick notes that he uses PATH_INFO for many PHP scripts like /index.php/go/do/something (where parameters are passed to PHP as "fake" subdirectories). He received a 404 Not Found errors with Apache 2 but not Apache 1.3.27. He had to add "AcceptPathInfo On" to file httpd.conf. For details, see http://httpd.apache.org/docs-2.2/mod/core.html#acceptpathinfo
• Using style tags in addition to style tags
If you want to also use the old-style (or MS ASP-like) tags then add this line to your php.ini file. This is the default starting with PHP 5:
short_open_tag = On
• GD library now built-in PHP
PHP 4.3.0 has gd built-in, just pass --with-gd to PHP configure.
• --with-apache
If you don't have Apache source at the default directory at /usr/local/apache, then use --with-apache=DIR with PHP configure, where DIR is the root source directory, This tells PHP where to look for the Apache source.
• Redhat 9
Apache 2 and PHP is built-in RedHat 9 (packages httpd 2.0.40, and php 4.2.2). Apache is mostly under /usr/lib/httpd and /usr/sbin/httpd. PHP is at /usr/lib/httpd/modules/libphp4.so For a complete file listing, type: rpm -ql httpd php
For RedHat 9, PHP 4.3.1 gives a compiler errors with RedHat 9 similar to: my_malloc.c:24: undefined reference to `errno'
The problem is with mysql and will have to wait for a fix from them. Until then, add this line:
#include
to the beginning of file ./ext/mysql/libmysql/mysql.h in your PHP source and remake PHP (from scratch).
• Notes about Gentoo Linux and Apache2 with PHP
LoadModule php5_module modules/libphp5.so
becomes:
LoadModule php5_module /usr/lib/apache2-extramodules/libphp5.so
To start automatically under Gentoo:
rc-update add apache2 default
(change php5 to php4 for PHP 4)
The following notes are how I got Apache 2 and PHP 5 (or PHP 4) working together on Linux. These instructions also apply, mostly, for any UNIX-like system, especially other Linux distributions. If you have a recent Linux distribution (say since 2002), you already have Apache 2 and PHP, so you don't need to do this unless you want the latest Apache 2 or PHP release or need to customize the Apache or PHP software. Don't forget to remove (or at least disable) the Apache rpm package if you install your own custom Apache.
Apache 2 Version Tip: Beginning with Apache 2.0.42 the API will be kept stable (yeah!). That means you will NOT have to recompile modules (and possibly upgrade/fix source) every time you install a new Apache release. This assumes you stay in the same Apache release series. For example, upgrading from 2.2.0 to 2.2.2 should work. This will not apply to upgrading to the next series (e.g., "development" 2.3.x or "stable" 2.4.x).
I was only able to get PHP working with Apache 2 as a *.so DSO ("dynamic shared object"), as opposed to compiled into Apache (with a static *.a library). I think DSO is the only way PHP is supported now on Apache 2.
I first used httpd-2.0.43 and php-4.3.0 with RedHat 7.3. I am now using httpd-2.2.3 and php-5.2.0 with SUSE 10.1.
Note: If you have problems with PHP and think it's a recent bug, you may want to consider using the latest http://snaps.php.net/ snapshot. Beware that snapshots frequently have regression and are not for production use. Usually problems are because of mis-configuration, not bugs, so snapshots will probably hurt more than help.
1. Download/unpack Apache2 source from the Apache httpd server website, http://httpd.apache.org/
2. In the Apache 2 source directory, create a Makefile by typing:
./configure --prefix=/usr/local/apache \
--enable-so \
--enable-cgi \
--enable-info \
--enable-rewrite \
--enable-speling \
--enable-usertrack \
--enable-deflate \
--enable-ssl \
--enable-mime-magic
You only need the enable-so line above. For information on other options, type ./configure --help and see "Compiling and Installing" in the Apache 2 Documentation, http://httpd.apache.org/
3. Make Apache from the just-created Makefile:
make
4. If make is successful, install Apache as root:
make install
5. Download/unpack PHP source from the PHP website, http://www.php.net/
Pick the latest from the 4.x series or 5.x series.
6. In the PHP source directory, create a Makefile by typing:
./configure \
--with-apxs2=/usr/local/apache/bin/apxs \
--with-mysql \
--prefix=/usr/local/apache/php \
--with-config-file-path=/usr/local/apache/php \
--enable-force-cgi-redirect \
--disable-cgi \
--with-zlib \
--with-gettext \
--with-gdbm
You only need the --with-apxs2, and prefix lines. --with-mysql adds MySql, --with-config-file moves the php.ini file location, disable-cgi disables the CGI version, which is not needed if you use Apache modules. It also enables and installs the command line interface (CLI) version. --with-zlib allows use of gzip-type compression, --with-gettext is for internationalization, and --with-gdbm allows access to GDBM databases. For more information, type ./configure --help and see the "Installation" chapter in the PHP Manual, http://ww.php.net/docs.php
7. Make PHP from the just-created Makefile:
make
8. If make is successful, type this as root to install PHP:
make install
If you are not root (I do not perform makes while root, for security and safety reasons), become root and type the following:
make install-su
9. If file /usr/local/apache/modules/libphp5.so does not exist or is an older version, type this (change this to libphp4.so for PHP 4):
cp -p .libs/libphp5.so /usr/local/apache/modules
10. Install the php.ini file:
cp -p php.ini-recommended /usr/local/apache/php/php.ini
11. Add these directives are in /usr/local/apache/conf/httpd.conf (if already there, verify they are correct):
# Make sure there's only **1** line for each of these 2 directives:
# Use for PHP 4.x:
#LoadModule php4_module modules/libphp4.so
#AddHandler php-script php
# Use for PHP 5.x:
LoadModule php5_module modules/libphp5.so
AddHandler php5-script php
# Add index.php to your DirectoryIndex line:
DirectoryIndex index.html index.php
AddType text/html php
# PHP Syntax Coloring
# (optional but useful for reading PHP source for debugging):
AddType application/x-httpd-php-source phps
Note 1: The php documentation recommends AddType application/x-httpd-php php instead of the above. However, it causes problems with the "MultiViews" feature of HTTP. That is, if the .php extension is left off a URL, and with certain browser Accept headers, Apache will not know .php (application/x-httpd-php) is HTML and will return a 406 Not Acceptable error. Using the AddType and AddHandler as shown above fixes this problem. For details see Mark Tranchant's webpage, "Using Apache's MultiViews with PHP whilst avoid 406 errors," and PHP bug 28023.
Note 2: PHP Syntax coloring isn't required, but it's very nice for looking at your php source while debugging. Here's an example.
Note 3: just for completeness I'll mention that you should be able to use SetOutputFilter / SetInputFilter instead of AddType, but you can't use both. However, SetOutputFilter / SetInputFilter no longer works for me. It used to work with an earlier PHP 4.x or Apache 2 version, but not with Apache 2.0.47/PHP 4.3.3. I understand this (PHP as an Apache 2 filter) is experimental, so I don't use it anymore:
SetOutputFilter PHP
SetInputFilter PHP
12. You're now ready to try it out. Start Apache (httpd) as root:
/usr/local/apache/bin/apachectl start
13. Perform these sanity checks to verify your install went OK:
$ /usr/local/apache/bin/httpd -t
Syntax OK
$ /usr/local/apache/bin/httpd -v
Server version: Apache/2.2.2
Server built: May 29 2006 12:40:55
$ /usr/local/apache/bin/httpd -V
Server version: Apache/2.2.2
Server built: May 29 2006 12:40:55
Server's Module Magic Number: 20051115:2
Server loaded: APR 1.2.7, APR-Util 1.2.7
Compiled using: APR 1.2.7, APR-Util 1.2.7
Architecture: 32-bit
Server MPM: Prefork
threaded: no
forked: yes (variable process count)
Server compiled with....
-D APACHE_MPM_DIR="server/mpm/prefork"
-D APR_HAS_SENDFILE
-D APR_HAS_MMAP
-D APR_HAVE_IPV6 (IPv4-mapped addresses enabled)
-D APR_USE_SYSVSEM_SERIALIZE
-D APR_USE_PTHREAD_SERIALIZE
-D SINGLE_LISTEN_UNSERIALIZED_ACCEPT
-D APR_HAS_OTHER_CHILD
-D AP_HAVE_RELIABLE_PIPED_LOGS
-D DYNAMIC_MODULE_LIMIT=128
-D HTTPD_ROOT="/usr/local/apache"
-D SUEXEC_BIN="/usr/local/apache/bin/suexec"
-D DEFAULT_PIDLOG="logs/httpd.pid"
-D DEFAULT_SCOREBOARD="logs/apache_runtime_status"
-D DEFAULT_LOCKFILE="logs/accept.lock"
-D DEFAULT_ERRORLOG="logs/error_log"
-D AP_TYPES_CONFIG_FILE="conf/mime.types"
-D SERVER_CONFIG_FILE="conf/httpd.conf"
$ /usr/local/apache/bin/httpd -S
VirtualHost configuration:
. . .
$ /usr/local/apache/bin/httpd -l
Compiled in modules:
core.c
. . .
mod_so.c
$ /usr/local/apache/bin/httpd -M
Loaded Modules:
. . .
php5_module (shared)
Syntax OK
(the above works for Apache 2.2.x and higher only)
$ ps -ef |grep httpd
root 24069 1 0 09:17 ? 00:00:08 /usr/local/apache/bin/httpd -k s
apache 29917 24069 0 15:30 ? 00:00:00 /usr/local/apache/bin/httpd -k s
. . .
Note: on BSD-based UNIX systems, you need to use "ps -aux" instead of "ps -ef".
14. Access your webserver with telnet. Type HEAD / HTTP/1.0 followed by a blank line:
$ telnet localhost 80
Trying 127.0.0.1...
Connected to localhost (127.0.0.1).
Escape character is '^]'.
HEAD / HTTP/1.0
HTTP/1.1 200 OK
Date: Mon, 29 May 2006 23:28:18 GMT
Server: Apache/2.2.2 (Unix) mod_ssl/2.2.2 OpenSSL/0.9.7a PHP/5.1.4
X-Powered-By: PHP/5.1.4
Last-Modified: Wed, 15 Mar 2006 06:53:17 GMT
Vary: Accept-Encoding
Connection: close
Content-Type: text/html; charset=ISO-8859-1
Content-Language: en
Connection closed by foreign host.
15. Access your webserver with your favorite browser. The following is a good test page to use for PHP. You only need the one line in bold is needed to display PHP configuration information. Name the file anything you want, but it must end with .php, such as phpinfo.php, and move the file to your web server content directory (for me /usr/local/apache/htdocs), with read permission set:
Tips and Notes
• Disabling Existing Apache Software
If Apache is already installed with your Linux distribution you have to disable it before installing your own custom version. To do this, type the following under RedHat Linux:
/sbin/chkconfig --del httpd
/etc/init.d/httpd stop
You may also remove the httpd and php rpm packages, and dependent packages, if you wish.
• Automatic Startup of Apache
To start Apache automatically, follow these steps:
● Copy Apache startup file to the startup directory:
cp /usr/local/apache/bin/apachectl /etc/init.d/
● Edit /etc/init.d/apachectl by inserting these 2 lines in bold:
#!/bin/sh
#
# chkconfig: - 85 15
# description: Apache is a Web server used to serve HTML files and CGI.
#
# Copyright 2000-2005 The Apache Software Foundation or its licensors, as
# applicable.
. . .
# Apache control script designed to allow an easy command line interface
# to controlling Apache. Written by Marc Slemko, 1997/08/23
. . .
● Enable httpd to startup automatically:
/sbin/chkconfig --add apachectl
/sbin/chkconfig --level 2 apachectl on
This works for RedHat Linux, Mandrake, SuSE, and Sun Java Desktop System. The equivalent of chkconfig for Debian is:
update-rc.d apachectl defaults
For other distributions, make appropriate softlinks to directories /etc/rc*.d/
● Reboot Linux to verify Apache starts
[Thanks to Beriah Dutcher and others]
• Make doesn't make dynamic library libphp5.so, just static library libphp5.a
First make sure libphp5.so isn't there. It's usually in the libs or .libs subdirectory. Run find from the PHP source directory root: find . -name libphp5.so If that doesn't work, make sure you use the --with-apxs2 option with the correct directory in ./configure. For example:
./configure --with-mysql --with-apxs2=/path/to/apache/apxs
When re-running ./configure always start with a fresh, clean source directory (no object).
(For PHP 4, substitute libphp4.so and libphp4.a for libphp5.so and libphp5.a, respectively.)
• PHP 5: My PHP scripts that worked with PHP 4 now ignores input variables
If you have old PHP 4 scripts you don't want to convert to PHP 5, change this line in your php.ini file:
register_long_arrays = On
For the long term, you may want to upgrade your PHP software or convert the PHP source code to use new-style variables. E.g., use $_REQUEST[i] instead of $i
• PHP 5.2.0 Upgrade from 5.1.x (November 2006)
I had one problem with upgrading from PHP 5.1.x to 5.2.0. The Phorum 3 software I use has a function date_format that now conflicts with a new PHP 5.2 internal function, also called date_format. This new function is used for PHP 5.2.x's new "datetime support". I fixed this by renaming Phorum's date_format function to not conflict with PHP's function. See also PHP 5.2 Update Info for more PHP 5.2 upgrade information.
• PHP 5.x: you may need to upgrade libxml2 to use XML
To use PHP 5 and XML (--with-xml), you need the libxml2 library version 2.6.0 or greater. You may need to upgrade if you have an older Linux distribution. Check with rpm -q libxml2
If it's not installed, check your installation disks for the libxml2 package. If it's too old, you can get the library from XMLSoft.org. You will probably need to compile and install it yourself.
• PHP 4.3.5 / 4.3.6 / 5.0RC1 PCRE Regression (March 2004)
If you use PHP 4.3.5, 4.3.6, or PHP 5.0 RC1 with PCRE (Perl Regular Expressions, the default), you get a core dump when you try and restart (SIGHUP) Apache (at least with Apache 2.0.49). The cause is from adding a new version of PCRE, 4.5, in PHP. This is fixed in PHP 4.3.7 and 5.0.
The symptom are these messages in error_log (which is also a symptom for lots of other memory-type bugs in Apache):
[Wed Mar 31 17:14:43 2004] [notice] SIGHUP received. Attempting to restart
[Wed Mar 31 17:14:43 2004] [notice] seg fault or similar nasty error detected in the parent process
[Wed Mar 31 17:14:48 2004] [warn] pid file /var/run/httpd.pid overwritten -- Unclean shutdown of previous Apache run?
To fix, upgrade to PHP 4.3.7 or higher, which fixes it (PHP 5.0 or higher also fixes the bug). A workaround is to stop/start Apache, instead of restarting Apache.
• ServerRoot Installation Directory
By default, Apache is installed in ServerRoot, as specified in httpd.conf or the httpd -d option:
ServerRoot "/usr/local/apache"
Some packagers and distributions move the location to /usr/local/httpd or /usr/local/apache2 so check that Apache is installed where you say it is, and check that you don't have two separate Apache installations by mistake.
• PHP-required Apache files must be present
PHP expects certain files to be present. They may be missing for some Linux distributions, which tend to separate files among the /var, /etc, and other directories. The apxs program expects to find httpd under the bin directory where apache is installed (by default /usr/local/apache/bin). PHP expects to find the os.h and unixd.h header files under the include directory where apache is installed (by default /usr/local/apache/include).
• MySQL
Use the --with-mysql configure option to build PHP with PHP's version of MySql. If you want the system's MySQL, or are also using MySql with another apache module (e.g., mod_perl), use this: with-mysql=/usr
• Custom OpenSSL and SSL
When using "--enable-ssl" (i.e, build with OpenSSL), the configure script will try to guess the location of OpenSSL. If you're using your own custom SSL, you must specify the location. For example, "--with-ssl=/usr/local/ssl" (of course, you can use the SSL and PHP that comes with Redhat 9 and other distributions).
Please note that mod_ssl is only for Apache 1.x. Apache 2.x does not use mod_ssl—SSL is built into Apache 2.x.
Jason reports that with RedHat 9 on his system, the make failed at mod_ssl. It needed a missing krb5.h header file. The fix is to Add an environment variable, $CFLAGS, before making PHP. That is:
"-I/usr/kerberos/include/ -L/usr/kerberos/lib"
• gdbm
If you get this error when using "--with-gdbm":
configure: error: DBA: Could not find necessary header file(s).
then you are probably missing the gdbm-devel rpm package.
• Apache 1.x and Apache 2 Configuration Directives
The "AddModule," "AgentLog," "ReferrerLog," "RefererIgnore," "BindAddress," "ClearModuleList," "Port," "FancyIndexing," "ServerType," "AccessConfig," and "ResourceConfig" directives in Apache 1.x no longer exist in Apache 2. Also, "HAVE_*" httpd.conf definitions no longer exist. Comment them out of your httpd.conf for Apache 2. Most of these directives have been replaced by other directive(s).
• Error after glibc library upgrade
Apache should still work after upgrading glibc with security and bug fixes for the same release. If you use another architecture (say from or to i386/i686), you need to recompile. One symptom is an error like this in your Apache error_log:
[error] (38)Function not implemented:
• make clean
Kåre Olaussen adds this note: If you're compiling php4 for both Apache 1.3 and Apache 2, use a separate php source directory for each version of Apache. "The module wouldn't load into Apache 2, because I had forgotten to run make clean in between." [Ed. note: I recommend untaring the source from scratch in another directory, that way you are sure no old state information is left behind from previous makes. Do this for any configuration or version changes.]
• Enabling register_globals for Apache 2
From Duke: For those of you that are desperately trying to enable register_globals on a per-directory basis using Apache 2 with PHP 4 [or PHP 5] using .htaccess files, the syntax is different from Apache 1.x. In the directory you want to enable register_globals (or pretty much any other on/off setting), add an .htaccess file containing:
php_value register_globals 1
Note that php_flag does not work with Apache 2. Neither does on / off — you must use 0 (off) or 1 (on).
[Editor's note: To enable globally, add register_globals=off to your php.ini file.]
To enable register_globals on a per-directory basis, you can add to a
• Multi-Processing Module (MPM): don't use multi-threading MPMs
MPMs is how Apache 2 handles multiple web server requests. That is, with multiple processes or multiple threads or some combination. For now, on Linux (and even UNIX) you should only use the (default) prefork module with PHP. This is specified at compile time. Other MPM modules (any involving threads) break PHP. This is partly because PHP uses a great number of external libraries, and many or most of them are not thread-safe or thread-aware. In any case, Linux 2.4 doesn't handle threads efficiently yet—multiple processes are better (this changes with Linux 2.6, or RedHat 9 with 2.6 threads backported to Linux 2.4).
• API module structure `php5_module' in file . . . is garbled
If you get a message like this:
httpd: Syntax error on line 195 of /usr/local/apache/conf/httpd.conf: API module
structure `php5_module' in file /usr/local/apache/modules/libphp5.so is garbled
- perhaps this is not an Apache module DSO
it's because your version of PHP wasn't built for the version of Apache you are using (For PHP 4, substitute php4_module . . . libphp4.so). Recompile PHP in a "fresh" source directory (where you haven't built PHP before) and make sure --prefix, --with-config-file and --with-apxs2 points to the correct Apache directories you are using and the libphp4.so file has been copied to your /usr/local/apache/modules directory.
• PATH_INFO
Dan Fitzpatrick notes that he uses PATH_INFO for many PHP scripts like /index.php/go/do/something (where parameters are passed to PHP as "fake" subdirectories). He received a 404 Not Found errors with Apache 2 but not Apache 1.3.27. He had to add "AcceptPathInfo On" to file httpd.conf. For details, see http://httpd.apache.org/docs-2.2/mod/core.html#acceptpathinfo
• Using style tags in addition to style tags
If you want to also use the old-style (or MS ASP-like) tags then add this line to your php.ini file. This is the default starting with PHP 5:
short_open_tag = On
• GD library now built-in PHP
PHP 4.3.0 has gd built-in, just pass --with-gd to PHP configure.
• --with-apache
If you don't have Apache source at the default directory at /usr/local/apache, then use --with-apache=DIR with PHP configure, where DIR is the root source directory, This tells PHP where to look for the Apache source.
• Redhat 9
Apache 2 and PHP is built-in RedHat 9 (packages httpd 2.0.40, and php 4.2.2). Apache is mostly under /usr/lib/httpd and /usr/sbin/httpd. PHP is at /usr/lib/httpd/modules/libphp4.so For a complete file listing, type: rpm -ql httpd php
For RedHat 9, PHP 4.3.1 gives a compiler errors with RedHat 9 similar to: my_malloc.c:24: undefined reference to `errno'
The problem is with mysql and will have to wait for a fix from them. Until then, add this line:
#include
to the beginning of file ./ext/mysql/libmysql/mysql.h in your PHP source and remake PHP (from scratch).
• Notes about Gentoo Linux and Apache2 with PHP
LoadModule php5_module modules/libphp5.so
becomes:
LoadModule php5_module /usr/lib/apache2-extramodules/libphp5.so
To start automatically under Gentoo:
rc-update add apache2 default
(change php5 to php4 for PHP 4)
Web Server
Web Server
Introduction
Apache is probably the most popular Linux-based Web server application in use. Once you have DNS correctly setup and your server has access to the Internet, you'll need to configure Apache to accept surfers wanting to access your Web site.
This chapter explains how to configure Apache in a number of commonly encountered scenarios for small web sites.
Download and Install The Apache Package
Most RedHat and Fedora Linux software products are available in the RPM format. When searching for the file, remember that the Apache RPM's filename usually starts with the word httpd followed by a version number, as in httpd-2.0.48-1.2.rpm. It is best to use the latest version of Apache. (For more on RPMs, see Chapter 6, "Installing Linux Software").
When searching for the file, remember that the Redhat / Fedora Apache RPM package's filename usually starts with the word httpd followed by a version number, as in httpd-2.0.48-1.2.rpm. With Ubuntu / Debian the package name will have the apache prefix instead.
Note: Unless otherwise stated, the sample configurations covered in this chapter will be for Redhat / Fedora distributions. If you use Debian / Ubuntu, don’t worry, there will be annotations to make you aware of the differences.
How To Get Apache Started
Setting up the Apache server is easy to do, but the procedure differs between Linux distributions.
Redhat / Fedora
Use the chkconfig command to configure Apache to start at boot:
[root@bigboy tmp]# chkconfig httpd on
Use the httpd
Introduction
Apache is probably the most popular Linux-based Web server application in use. Once you have DNS correctly setup and your server has access to the Internet, you'll need to configure Apache to accept surfers wanting to access your Web site.
This chapter explains how to configure Apache in a number of commonly encountered scenarios for small web sites.
Download and Install The Apache Package
Most RedHat and Fedora Linux software products are available in the RPM format. When searching for the file, remember that the Apache RPM's filename usually starts with the word httpd followed by a version number, as in httpd-2.0.48-1.2.rpm. It is best to use the latest version of Apache. (For more on RPMs, see Chapter 6, "Installing Linux Software").
When searching for the file, remember that the Redhat / Fedora Apache RPM package's filename usually starts with the word httpd followed by a version number, as in httpd-2.0.48-1.2.rpm. With Ubuntu / Debian the package name will have the apache prefix instead.
Note: Unless otherwise stated, the sample configurations covered in this chapter will be for Redhat / Fedora distributions. If you use Debian / Ubuntu, don’t worry, there will be annotations to make you aware of the differences.
How To Get Apache Started
Setting up the Apache server is easy to do, but the procedure differs between Linux distributions.
Redhat / Fedora
Use the chkconfig command to configure Apache to start at boot:
[root@bigboy tmp]# chkconfig httpd on
Use the httpd
init script in the /etc/init.d directory to start,stop, and restart Apache after booting:
[root@bigboy tmp]# /etc/init.d/httpd start
[root@bigboy tmp]# /etc/init.d/httpd stop
[root@bigboy tmp]# /etc/init.d/httpd restart
You can test whether the Apache process is running with
[root@bigboy tmp]# pgrep httpd
you should get a response of plain old process ID numbers.
General Configuration Steps
The configuration file used by Apache is /etc/httpd/conf/httpd.conf in Redhat / Fedora distributions and /etc/apache*/httpd.conf in Debian / Ubuntu distributions. As for most Linux applications, you must restart Apache before changes to this configuration file take effect.
Where To Put Your Web Pages
All the statements that define the features of each web site are grouped together inside their own section, or container, in the httpd.conf file. The most commonly used statements, or directives, inside a container are:
● servername: Defines the name of the website managed by the container. This is needed in named virtual hosting only, as I'll explain soon.
● DocumentRoot: Defines the directory in which the web pages for the site can be found.
By default, Apache searches the DocumentRoot directory for an index, or home, page named index.html. So for example, if you have a servername of www.my-site.com with a DocumentRoot directory of /home/www/site1/, Apache displays the contents of the file /home/www/site1/index.html when you enter http://www.my-site.com in your browser.
Some editors, such as Microsoft FrontPage, create files with an .htm extension, not .html. This isn't usually a problem if all your HTML files have hyperlinks pointing to files ending in .htm as FrontPage does. The problem occurs with Apache not recognizing the topmost index.htm page. The easiest solution is to create a symbolic link (known as a shortcut to Windows users) called index.html pointing to the file index.htm. This then enables you to edit or copy the file index.htm with index.html being updated automatically. You'll almost never have to worry about index.html and Apache again!
This example creates a symbolic link to index.html in the /home/www/site1 directory.
[root@bigboy tmp]# cd /home/www/site1
[root@bigboy site1]# ln -s index.htm index.html
[root@bigboy site1]# ll index.*
-rw-rw-r-- 1 root root 48590 Jun 18 23:43 index.htm
lrwxrwxrwx 1 root root 9 Jun 21 18:05 index.html -> index.htm
[root@bigboy site1]#
The l at the very beginning of the index.html entry signifies a link and the -> the link target.
The Default File Location
By default, Apache expects to find all its web page files in the /var/www/html/ directory with a generic DocumentRoot statement at the beginning of httpd.conf. The examples in this chapter use the /home/www directory to illustrate how you can place them in other locations successfully.
File Permissions And Apache
Apache will display Web page files as long as they are world readable. You have to make sure you make all the files and subdirectories in your DocumentRoot have the correct permissions.
It is a good idea to have the files owned by a nonprivileged user so that Web developers can update the files using FTP or SCP without requiring the root password.
To do this:
1. Create a user with a home directory of /home/www.
2. Recursively change the file ownership permissions of the /home/www directory and all its subdirectories.
3. Change the permissions on the /home/www directory to 755, which allows all users, including the Apache's httpd daemon, to read the files inside.
[root@bigboy tmp]# useradd -g users www
[root@bigboy tmp]# chown -R www:users /home/www
[root@bigboy tmp]# chmod 755 /home/www
Now we test for the new ownership with the ll command.
[root@bigboy tmp]# ll /home/www/site1/index.*
-rw-rw-r-- 1 www users 48590 Jun 25 23:43 index.htm
lrwxrwxrwx 1 www users 9 Jun 25 18:05 index.html -> index.htm
[root@bigboy tmp]#
Note: Be sure to FTP or SCP new files to your web server as this new user. This will make all the transferred files automatically have the correct ownership.
If you browse your Web site after configuring Apache and get a "403 Forbidden" permissions-related error on your screen, then your files or directories under your DocumentRoot most likely have incorrect permissions. Appendix II, "Codes, Scripts, and Configurations," has a short script that you can use to recursively set the file permissions in a directory to match those expected by Apache. You may also have to use the Directory directive to make Apache serve the pages once the file permissions have been correctly set. If you have your files in the default /home/www directory then this second step becomes unnecessary.
user root creates a directory /home/www/site1 in which the pages for a new Web site will be placed. Using the ls -Z command, you can see that the user_home_t security label has been assigned to the directory and the index.html page created in it. This label is not accessible by Apache.
[root@bigboy tmp]# mkdir /home/www/site1
[root@bigboy tmp]# ls -Z /home/www/
drwxr-xr-x root root root:object_r:user_home_t site1
[root@bigboy tmp]# touch /home/www/site1/index.html
[root@bigboy tmp]# ls -Z /home/www/site1/index.html
-rw-r--r-- root root root:object_r:user_home_t /home/www/site1/index.html
[root@bigboy tmp]#
Accessing the index.html file via a Web browser gets a "Forbidden 403" error on your screen, even though the permissions are correct. Viewing the /var/log/httpd/error_log gives a "Permission Denied" message and the /var/log/messages file shows kernel audit errors.
[root@bigboy tmp]# tail /var/log/httpd/error_log
[Fri Dec 24 17:59:24 2004] [error] [client 216.10.119.250] (13)Permission denied: access to / denied
[root@bigboy tmp]# tail /var/log/messages
Dec 24 17:59:24 bigboy kernel: audit(1103939964.444:0): avc: denied { getattr } for pid=2188 exe=/usr/sbin/httpd path=/home/www/site1 dev=hda5 ino=73659 scontext=system_u:system_r:httpd_t tcontext=root:object_r:user_home_t tclass=dir
[root@bigboy tmp]#
Security Contexts For CGI Scripts
You can use Apache to trigger the execution of programs called Common Gateway Interface (CGI) scripts. CGI scripts can be written in a variety of languages, including PERL and PHP, and can be used to do such things as generate new Web page output or update data files. A Web page's Submit button usually has a CGI script lurking somewhere beneath. By default, CGI scripts are placed in the /var/www/cgi-bin/ directory as defined by the ScriptAlias directive you'll find in the httpd.conf file, which I'll discuss in more detail later.
ScriptAlias /cgi-bin/ "/var/www/cgi-bin/"
In the default case, any URL with the string /cgi-bin/ will trigger Apache to search for an equivalent executable file in this directory. So, for example, the URL, http://192.168.1.100/cgi-bin/test/test.cgi actually executes the script file /var/www/cgi-bin/test/test.cgi.
SELinux contexts have to be modified according to the values in Table 20.1 for a CGI script to be run in another directory or to access data files. In the example case, the PERL script test.cgi was created to display the word "Success" on the screen of your Web browser.
#!/usr/bin/perl
# CGI Script "test.cgi"
print qq(
#
#
#
#
#
#
#
Success!
#
#”
);
The ScriptAlias directive has been set to point to /home/www/cgi-bin/ instead of /var/www/cgi-bin/.
ScriptAlias /cgi-bin/ "/home/www/cgi-bin/"
User root creates the /home/www/cgi-bin/ directory, changes the directory's security context label to httpd_sys_script_exec_t, and then creates the script /home/www/cgi-bin/test/test.cgi mentioned previously with the correct executable file permissions.
[root@bigboy tmp]# mkdir -p /home/www/cgi-bin/test
[root@bigboy tmp]# chcon -h -t httpd_sys_script_exec_t /home/www/cgi-bin/
[root@bigboy tmp]# mkdir /home/www/cgi-bin/test
[root@bigboy tmp]# ls -Z /home/www/cgi-bin
drwxr-xr-x root root root:object_r:httpd_sys_script_exec_t test
[root@bigboy tmp]# vi /home/www/cgi-bin/test/test.cgi
[root@bigboy tmp]# chmod o+x /home/www/cgi-bin/test/test.cgi
[root@bigboy tmp]#
Accessing the URL http://192.168.1.100/cgi-bin/test/test.cgi is successful. Problems occur when the same test.cgi file needs to be used by a second Web site housed on the same Web server. The file is copied to a directory /web/cgi-bin/site2/ governed by the ScriptAlias in the second Web site's container (explained later), but the security context label isn't copied along with it.
ScriptAlias /cgi-bin/ "/web/cgi-bin/site2/"
The file inherits the context of its new parent.
[root@bigboy tmp]# cp /home/www/cgi-bin/test/test.cgi /web/cgi-bin/site2/test.cgi
[root@bigboy tmp]# ls -Z /web/cgi-bin/site2/test.cgi
-rw-r--r-x root root root:object_r:tmp_t /web/cgi-bin/site2/test.cgi
[root@bigboy tmp]#
Permission denied and kernel audit errors occur once more; you can fix them only by changing the security context of the test.cgi file.
[root@bigboy tmp]# tail /var/log/httpd/error_log
[Fri Dec 24 18:36:08 2004] [error] [client 216.10.119.250] (13)Permission denied: access to /cgi-bin/texcelon/test.cgi denied
[root@bigboy tmp]# tail /var/log/messages
Dec 24 18:36:08 bigboy kernel: audit(1103942168.549:0): avc: denied { getattr } for pid=2191 exe=/usr/sbin/httpd path=/web/cgi-bin/site2/test.cgi dev=hda5 ino=77491 scontext=system_u:system_r:httpd_t tcontext=root:object_r:tmp_t tclass=file
[root@bigboy tmp]#
Note: If you find security contexts too restrictive, you can turn them off system wide by editing your /etc/selinux/config file, modifying the SELINUX parameter to disabled. SELinux will be disabled after your next reboot.
Named Virtual Hosting
You can make your Web server host more than one site per IP address by using Apache's named virtual hosting feature. You use the NameVirtualHost directive in the /etc/httpd/conf/httpd.conf file to tell Apache which IP addresses will participate in this feature.
The containers in the file then tell Apache where it should look for the Web pages used on each Web site. You must specify the IP address for which each container applies.
Named Virtual Hosting Example
Consider an example in which the server is configured to provide content on 97.158.253.26. In the code that follows, notice that within each container you specify the primary Web site domain name for that IP address with the ServerName directive. The DocumentRoot directive defines the directory that contains the index page for that site.
You can also list secondary domain names that will serve the same content as the primary ServerName using the ServerAlias directive.
Apache searches for a perfect match of NameVirtualHost, , and ServerName when making a decision as to which content to send to the remote user's Web browser. If there is no match, then Apache uses the first in the list that matches the target IP address of the request.
This is why the first statement contains an asterisk: to indicate it should be used for all other Web queries.
NameVirtualHost 97.158.253.26
Default Directives. (In other words, not site #1 or site #2)
servername www.my-site.com
Directives for site #1
servername www.another-site.com
Directives for site #2
Be careful with using the asterisk in other containers. A with a specific IP address always gets higher priority than a statement with an * intended to cover the same IP address, even if the ServerName directive doesn't match. To get consistent results, try to limit the use of your statements to the beginning of the list to cover any other IP addresses your server may have.
You can also have multiple NameVirtualHost directives, each with a single IP address, in cases where your Web server has more than one IP address.
IP-Based Virtual Hosting
The other virtual hosting option is to have one IP address per Web site, which is also known as IP-based virtual hosting. In this case, you will not have a NameVirtualHost directive for the IP address, and you must only have a single container per IP address.
Also, because there is only one Web site per IP address, the ServerName directive isn't needed in each container, unlike in named virtual hosting.
IP Virtual Hosting Example: Single Wild Card
In this example, Apache listens on all interfaces, but gives the same content. Apache displays the content in the first directive even if you add another right after it. Apache also seems to enforce the single container per IP address requirement by ignoring any ServerName directives you may use inside it.
DocumentRoot /home/www/site1
IP Virtual Hosting Example: Wild Card and IP addresses
In this example, Apache listens on all interfaces, but gives different content for addresses 97.158.253.26 and 97.158.253.27. Web surfers get the site1 content if they try to access the web server on any of its other IP addresses.
DocumentRoot /home/www/site1
DocumentRoot /home/www/site2
DocumentRoot /home/www/site3
Configuration - Multiple Sites And IP Addresses
To help you better understand the edits needed to configure the /etc/httpd/conf/httpd.conf file, I'll walk you through an example scenario. The parameters are:
● The web site's systems administrator previously created DNS entries for www.my-site.com, my-site.com, www.my-cool-site.com and www.default-site.com to map the IP address 97.158.253.26 on this web server. The domain www.another-site.com is also configured to point to alias IP address 97.158.253.27. The administrator wants to be able to get to www.test-site.com on all the IP addresses.
● Traffic to www.my-site.com, my-site.com, and www.my-cool-site.com must get content from subdirectory site2. Hitting these URLs causes Apache to display the contents of file index.html in this directory.
● Traffic to www.test-site.com must get content from subdirectory site3.
● Named virtual hosting will be required for 97.158.253.26 as in this case we have a single IP address serving different content for a variety of domains. A NameVirtualHost directive for 97.158.253.26 is therefore required.
● Traffic going to www.another-site.com will get content from directory site4.
● All other domains pointing to this server that don't have a matching ServerName directive will get Web pages from the directory defined in the very first container: directory site1. Site www.default-site.com falls in this category.
How do these requirements translate into code? Here is a sample snippet of a working httpd.conf file:
ServerName localhost
NameVirtualHost 97.158.253.26
NameVirtualHost 97.158.253.27
#
# Match a webpage directory with each website
#
DocumentRoot /home/www/site1
DocumentRoot /home/www/site2
ServerName www.my-site.com
ServerAlias my-site.com, www.my-cool-site.com
DocumentRoot /home/www/site3
ServerName www.test-site.com
DocumentRoot /home/www/site4
ServerName www.another-site.com
#
# Make sure the directories specified above
# have restricted access to read-only.
#
Order allow,deny
Allow from all
AllowOverride FileInfo AuthConfig Limit
Options MultiViews Indexes SymLinksIfOwnerMatch IncludesNoExec
Order allow,deny
Allow from all
Order deny,allow
Deny from all
These statements would normally be found at the very bottom of the file where the virtual hosting statements reside. The last section of this configuration snippet has some additional statements to ensure read-only access to your Web pages with the exception of Web-based forms using POSTs (pages with "submit" buttons). Remember to restart Apache every time you update the httpd.conf file for the changes to take effect on the running process.
Note: You will have to configure your DNS server to point to the correct IP address used for each of the Web sites you host. Chapter 18, "Configuring DNS", shows you how to configure multiple domains, such as my-site.com and another-site.com, on your DNS server.
The Apache configuration file is: /etc/httpd/conf/httpd.conf
Web pages are served from the directory as configured by the DocumentRoot directive. The default directory location is:
● Red Hat 7.x-9, Fedora Core, Red Hat Enterprise 4, CentOS 4: /var/www/html/
● Red Hat 6.x and older: /home/httpd/html/
● Suse 9.x: /srv/www/htdocs/
● Ubuntu (dapper 6.06) / Debian: /var/www/html
The default home page for the default configuration is index.html. Note the pages should not be owned by user apache as this is the process owner of the httpd web server daemon. If the web server process is comprimised, it should not be allowed to alter the files. The files should of course be readable by user apache.
Apache may be configured to run as a host for one web site in this fashion or it may be configured to serve for multiple domains. Serving for multiple domains may be achieved in two ways:
● Virtual hosts: One IP address but multiple domains - "Name based" virtual hosting.
● Multiple IP based virtual hosts: One IP address for each domain - "IP based" virtual hosting.
The default configuration will allow one to have multiple user accounts under one domain by using a reference to the user account: http://www.domain.com/~user1/. If no domain is registered or configured, the IP address may also be used: http://XXX.XXX.XXX.XXX/~user1/.
[Potential Pitfall] The default umask for directory creation is correct by default but if not use: chmod 755 /home/user1/public_html
[Potential Pitfall] When creating new "Directory" configuration directives, I found that placing them by the existing "Directory" directives to be a bad idea. It would not use the .htaccess file. This was because the statement defining the use of the .htaccess file was after the "Directory" statement. Previously in RH 6.x the files were separated and the order was defined a little different. I now place new "Directory" statements near the end of the file just before the "VirtualHost" statements.
For users of Red Hat 7.1, the GUI configuration tool apacheconf was introduced for the crowd who like to use pretty point and click tools.
Files used by Apache:
● Start/stop/restart script:
● Red Hat/Fedora/CentOS: /etc/rc.d/init.d/httpd
● SuSE 9.3: /etc/init.d/apache2
● Ubuntu (dapper 6.06) / Debian: /etc/init.d/apache2
● Apache main configuration file:
● Red Hat/Fedora/CentOS: /etc/httpd/conf/httpd.conf
● SuSE: /etc/apache2/httpd.conf
(Need to add directive: ServerName host-name)
● Ubuntu (dapper 6.06) / Debian: /etc/apache2/apache2.conf
● Apache suplementary configuration files:
● Red Hat/Fedora/CentOS: /etc/httpd/conf.d/component.conf
● SuSE: /etc/apache2/conf.d/component.conf
● Ubuntu (dapper 6.06) / Debian:
● Virtual domains: /etc/apache2/sites-enabled/domain
(Create soft link from /etc/apache2/sites-enabled/domain to /etc/apache2/sites-available/domain to turn on)
● Additional configuration directives: /etc/apache2/conf.d/
● Modules to load: /etc/apache2/mods-available/
(Soft link to /etc/apache2/mods-enabled/ to turn on)
● Ports to listen to: /etc/apache2/ports.conf
● /var/log/httpd/access_log and error_log - Red Hat/Fedora Core Apache log files
(Suse: /var/log/apache2/)
Start/Stop/Restart scripts: The script is to be run with the qualifiers start, stop, restart or status.
i.e. /etc/rc.d/init.d/httpd restart. A restart allows the web server to start again and read the configuration files to pick up any changes. To have this script invoked upon system boot issue the command chkconfig --add httpd.
Also Apache control tool: /usr/sbin/apachectl start
Apache Control Command: apachectl:
Red Hat / Fedora Core / CentOS: apachectl directive
Ubuntu dapper 6.06 / Debian: apache2ctl directive
Directive Description
start Start the Apache httpd daemon. Gives an error if it is already running.
stop Stops the Apache httpd daemon.
graceful Gracefully restarts the Apache httpd daemon. If the daemon is not running, it is started. This differs from a normal restart in that currently open connections are not aborted.
restart Restarts the Apache httpd daemon. If the daemon is not running, it is started. This command automatically checks the configuration files as in configtest before initiating the restart to make sure the daemon doesn't die.
status Displays a brief status report.
fullstatus Displays a full status report from mod_status. Requires mod_status enabled on your server and a text-based browser such as lynx available on your system. The URL used to access the status report can be set by editing the STATUSURL variable in the script.
configtest
-t Run a configuration file syntax test.
Apache Configuration Files:
● /etc/httpd/conf/httpd.conf: is used to configure Apache. In the past it was broken down into three files. These may now be all concatenated into one file.
● /etc/httpd/conf.d/application.conf: All configuration files in this directory are included during Apache start-up. Used to store application specific configurations.
● /etc/sysconfig/httpd: Holds environment variables used when starting Apache.
Basic settings: Change the default value for ServerName www.
Giving Apache access to the file system: It is prudent to limit Apache's view of the file system to only those directories necessary. This is done with the directory statement. Start by denying access to everything, then grant access to the necessary directories.
Deny access completely to file system root ("/") as the default:
Options None
AllowOverride None
Grant access to a user's directory:
AllowOverride None
order allow,deny
allow from all
Options Indexes Includes FollowSymLinks
OR
use the statement UserDir public_html which does this by default for every user account at $HOME/public_html. Change to a comment (add "#" at beginning of line) from Fedora Core default UserDir disable.
File permissions: The Apache web server daemon must be able to read your web pages in order to feed thier contents to the network. Use an appropriate umask and file protection. This works: chmod ugo+r -R public_html
One may also use groups to control permisions.
If the Apache web server can not access the file you will get the error "403 Forbidden" "You don't have permission to access file-name on this server." Note the default permissions on a user directory when first created with "useradd" are:
drwx------ 3 userx userx
You must allow the web server running as user "apache" to access the directory if it is to display pages held there.
Fix with command: chmod ugo+rx /home/userx
drwxr-xr-x 3 userx userx
Change Apache Port number:
"cat /path/to/httpd.conf grep 80"
if where is says 80 (default port) seems to be in context of wha you're doing... change it. httpd.conf
Change Apache Document Root:
cat /etc/httpd/conf/httpd.confgrep DocumentRoot
# DocumentRoot: The directory out of which you will serve your
DocumentRoot "/var/www/html"
# This should be changed to whatever you set DocumentRoot to.
# DocumentRoot /www/docs/dummy-host.example.com
Subscribe to:
Posts (Atom)