Archive for September, 2007

Added tag code to theloop.php

Thursday, September 27th, 2007

<?php if (function_exists('the_tags')) { ?>

<?php } ?>

goes right after <div class="entry-content"> to display tags for current post. I haven’t implemented it here yet, but it’s working on my test blog. I also edited out the conditional statement to make the tags only showup on single post pages, by deleting the if is_single() and changing the elseif to if. I had to correct the curly brackets, too.

Props to Rich Gilchrest for the inspiration.

Are you worried about a hacker stealing the logon details of people who have registered at your site?

Thursday, September 27th, 2007

Allow OpenID registration, and get out of the business of storing passwords. It’s more convenient for users, too.

This process isn’t totally ready for prime-time, but I can confirm it works on Wordpress 2.3.
All you need to do is download the wpopenid plugin, and install the phpopenid library. I put the Auth folder in wp-content, but you could put it anywhere and include that directory in your php path.

Go activate the plug-in and set the options. You’ll get some complaints about the database, because of the change in 2.3 databases, but it should still work.

You can try it out here.

Do you have too many passwords to remember?

Thursday, September 27th, 2007

Or worse, do you use the same password for everything? There’s a better way to do it, which relieves every site and blog owner from having to store your password.

The way to do it is to set up an OpenID, that way the only person you have to blame is yourself, and perhaps your hosting provider. Another benefit of doing this touches on a big interest of mine: owning your identity online. You see, in the OpenID scheme, your blog URL is all you provide upon login. The rest of the information is exchanged by encrypted API handshake between the OpenID provider(your site) and the site you’re logging into(the OpenID consumer).

The setup is not for the faint of heart, but it’s not that bad, and I took good notes on how I did it.

To set up your site as an OpenID provider, you need to following things:

  • A hosting account running PHP5 and offering shell access, and a domain name. I always recommend Dreamhost, and you can get hosting for only $5/month using my “Synthesis” promo code.

    Step 1: Download the latest and greatest version of the PHP script. It’s 0.6 currently.
    Log into your account, and open a shell window, and issue the following commands(if you’re not running PHP as CGI, you can skip the .htaccess part and do everything from the root URL if you want - read below to see how to tell):
    mkdir me
    cd me
    svn co https://www.siege.org/svn/oss/phpMyID/trunk/ .

    Step 2: Visit MyID.config.php in your web browser. You should see something like this:

    This is an OpenID server endpoint. For more information, see http://openid.net/
    Server: http://williamgunn.org/me/OpenID.config.php
    Realm: phpMyID
    Login

    Step 3: Edit MyID.config.php to include your info.
    - pick a username
    - generate and include your password hash
    echo -n 'username:realm:password' | openssl md5
    - optionally enter personal information to be supplied to sites.

    Step 4: Go back to your webbrowser and hit the login link on the page you loaded before. Now, you’ll be redirected and a window will popup asking for your username and password. Enter them, and press enter. If you get a message saying you’re logged in, you’re not running php as a cgi, and you can skip the rest of this paragraph. If you are running php as a cgi, you need a .htaccess file to convert the authentication headers into environment variables the script can use. This is because “The HTTP Authentication hooks in PHP are only available when it is running as an Apache module and is hence not available in the CGI version.” The fix is easy: if you didn’t make a subdirectory above, make one, and create a file called .htaccess containing the following code(you may already have an example file in your root directory).
    # Option 1, mod_rewrite (req)
    RewriteEngine on
    RewriteCond %{HTTP:Authorization} !^$
    RewriteCond %{QUERY_STRING} openid.mode=authorize
    RewriteCond %{QUERY_STRING} !auth=
    RewriteCond %{REQUEST_METHOD} =GET
    RewriteRule (.*) %{REQUEST_URI}?%{QUERY_STRING}&amp;auth=%{HTTP:Authorization} [L]

    Now, you should be able to hit the login link and get logged in.

    I get redirected to the following URL: http://williamgunn.org/me/scriptname.config.php?openid.mode=id_res&amp;openid.identity=http%3A%2F%2Fwilliamgunn.org%2Fme%2Fscriptname.config.php&amp;openid.assoc_handle=[redacted]&amp;openid.return_to=http%3A%2F%2Fwilliamgunn.org%2Fme%2Fscriptname.config.php&amp;openid.signed=mode%2Cidentity%2Cassoc_handle%2Creturn_to&amp;openid.sig=[more redacted stuff]

    Now set allow_gmp and allow_test to true in the config file, allowing encryption aka Smart Mode and testing, and load http://yoursite.com/you/MyID.config.php?openid.mode=test

    If you’re on Dreamhost, which doesn’t have support for GMP in the php binaries, you will get a file looking somewhat like this:

    bcmath warn - not loaded
    gmp warn - not loaded
    logfile warn - log is not writable
    session pass
    secret pass
    expire pass
    base64 pass
    hmac pass
    bigmath fail - big math functions are not available.
    sha1_20 pass
    x_or pass

    If you get that, you’re ready to delegate your OpenID identity. Go to the root of your domain(or wherever you wish) and enter the following code in a file named index.html
    <link href="http://williamgunn.org/me/OpenID.config.php" rel="openid.server" />
    <link href="http://williamgunn.org/me/OpenID.config.php" rel="openid.delegate" />

    If you’ve already got an index.html, just put the two lines in the head section.

    Now go login to a site that accepts OpenID, or just leave a comment here. See here for details on how to set up your site to accept OpenID, see here.

  • Write a Google sitemap for your Wordpress blog

    Friday, September 21st, 2007

    One of the most effective ways to increase the visibility of your content is to make sure it’s indexed regularly by Google. However, the Googlebot sometimes has a hard time with database-driven websites like Wordpress blogs, so it helps if you tell Google which URLs to visit. The way to do that is with an XML sitemap. There are a couple different kinds of sitemaps, which work with different search engines, but I’m only going to talk about the XML sitemap supported by Google and Yahoo. There’s also a Google sitemap generator for Wordpress, but if you’re like me, you try to keep the number of active plug-ins to a minimum to make your site as fast as possible.

    Not only will a sitemap ensure Google has the freshest content from your site, but it will also make your site run faster by telling the Googlebot that it doesn’t need to crawl your back archives with the same frequency as your front page. This is especially important for shared hosting situations like Dreamhost. Because the Googlebot alone can use 50% of the CPU of the shared server, if your site isn’t configured properly, you could bog down the server for everyone else and even get your site taken offline1.

    To set this up you’ll need an account with Google Webmaster Tools, the downloadable sitemap generator, and a hosting account that uses Analog logging and offers python support. I use Dreamhost. If you need a host, check ‘em out (and use promo code “Synthesis” to get your first year for $60).

    First, download the program and upload it to the base directory of your website. Unzip the package and open up config-example.xml. In config-example.xml are the parameters that control how the URL list that makes up the sitemap is generated. You’ll need to rename this to config.xml for it to work. There are two steps to setting up config.xml: Including URLs, and excluding URLs. Because sitemap_gen doesn’t do any crawling itself, you have to supply it with a list of URLs. One simple way to do this is with a text listing of URLs, but manually adding to this list every time you wrote a new post would get tedious. Conveniently, sitemap_gen can parse logfiles, so you can use your logs as the URL list. The frequency with which URLs appear in your logs also allows sitemap_gen to assign a priority score to each URL, letting the Googlebot know which pages to update more frequently and which pages it doesn’t need to crawl as often.

    Next, find the section in config.xml that says, “The “site” node describes your basic web site.” In this section, you want to replace http://www.example.com with the path to your site. Replace /var/www/docroot/sitemap.xml.gz or whatever comes after store_into with the name of your sitemap. I used sitemap.xml.gz, to generate a compressed sitemap for google to read.

    Moving down the file, find the INPUTS section. This is where you will specify which URLs to include in the sitemap. This part if broken up into sections which contain different link inclusion mechanisms. You can only use one mechanism at a time, so delete or comment out the sections until you get to the one that talks about accesslogs. Remove two of the three example statements in brackets in this section, and modify the remaining one to contain the full path to your access logs. You can use the * character to specify all the logs in the directory like so <accesslog path="/path/to/logs/access.log*" encoding="UTF-8" /> . Delete the remaining sections in the INPUTS section.

    The next section is the filters section. This is where you will specify which URLs to exclude. You can do a lot of fancy stuff here, but the most important thing for Wordpress is to remove URLs that lead to non-content pages, like wp-login, for example2. In these statements you tell sitemap_gen which URLs to add or remove from the list, using normal wildcards or regular expressions. I recommend keeping this as simple as possible, avoiding the use of pass statements because those act like short circuits and will leave matching URLs in the list no matter what you specify later, and in conjunction with regular expressions, this can sometimes be non-intuitive and hard to debug.

    Here’s my filters section:

    <filter action="drop"  type="regexp"  pattern="/wp-admin/"    />  
      <filter action="drop"  type="regexp"  pattern="/wp-login/"       /> 
      <filter action="drop"  type="regexp"  pattern="wp-cron\\.php"    />    
      <filter action="drop"  type="regexp"  pattern="wp-login\\.php"      />  
      <filter action="drop"  type="regexp"  pattern="/doc/"        />
      <filter action="drop"  type="regexp"  pattern="/noexist_" />       
      <filter action="drop"  type="regexp"  pattern="/\\?p=[\\d]"      />  
      <filter action="drop"  type="regexp"  pattern="/\\?s=[a-zA-Z0-9]" />       
      <filter action="drop"  type="regexp"  pattern="/Photos/tags/.*\\.html" />       
      <filter action="drop"  type="regexp"  pattern="/Photos/tags/.*/tags/"    />    
      <filter action="drop"  type="regexp"  pattern="/wp-content/"  />
      <filter action="drop"  type="regexp"  pattern="/wp-includes/" />
      <filter action="drop"  type="regexp"  pattern="/stats/" />
      <filter action="drop"  type="regexp"  pattern="/_vti_bin/" />
      <filter action="drop"  type="regexp"  pattern="/MSOffice/" />
      <filter action="drop"  type="regexp"  pattern="/dh_phpmyadmin/"/> 
      <filter action="drop"  type="regexp"  pattern="/htmledit/" />
      <filter action="drop"  type="regexp"  pattern="/robots\\.txt" />
      <filter action="drop"  type="regexp"  pattern="/sitemap\\.xml"/> 
      <filter action="drop"  type="regexp"  pattern="/xmlrpc\\.php" />
      <filter action="drop"  type="wildcard"  pattern="*.jpg"         />
      <filter action="drop"  type="wildcard"  pattern="*.tif"         />
      <filter action="drop"  type="wildcard"  pattern="*.tiff"        /> 
      <filter action="drop"  type="wildcard"  pattern="*.bmp"       />  
      <filter action="drop"  type="wildcard"  pattern="*.ico"         />
      <filter action="drop"  type="wildcard"  pattern="*.js"         />
      <filter action="drop"  type="wildcard"  pattern="*.css"       />  
      <filter action="drop"  type="wildcard"  pattern="*.gif"        /> 
         <!-- Exclude URLs within UNIX-style hidden files or directories       -->
      <filter action="drop"  type="regexp"    pattern="/\\.[^/]*"   />  
    

    That’s all fairly straightforward, I hope, but two things merit explaining. The section below

    <filter action="drop"  type="regexp"  pattern="/\\?p=[\\d]"     />   
      <filter action="drop"  type="regexp"  pattern="/\\?s=[a-zA-Z0-9]"    />    
      <filter action="drop"  type="regexp"  pattern="/Photos/tags/.*\\.html"    />    
      <filter action="drop"  type="regexp"  pattern="/Photos/tags/.*/tags/"     />   
    

    is an example of one way to remove redundant URLs from your list. You don’t need the “Pretty URL” to your site and the /p?number URL both, and if you’ve changed that setting recently, they will both show up in your logs. The /\?p=[\d] string tells site_gen to exclude any URL of the form /p?some number. Also, you don’t necessarily need search result pages to appear in the list, so the next line takes care of that. The following two lines are for use with the Flickr Photo Gallery plugin. This plugin allows you to browse your tags just as you would at Flickr, but this creates a URL problem when the site is crawled, resulting in 90% of your logs being composed of redundant crap. Those two lines remove all the URLs pertaining to the gallery except gallery pages and display pages for a single tag.

    The next thing worth mentioning is the lines below, which are generated when someone using IE visits your page with the discussion toolbar loaded. IE looks to see if your site supports it, which mine doesn’t.

    <filter action="drop" type="regexp" pattern="/_vti_bin/" />
    <filter action="drop" type="regexp" pattern="/MSOffice/" />

    After processing your logs and applying some intelligent filter rules to exclude URLs that aren’t content-containing parts of your site, you’re ready to submit. Run python sitemap_gen.py --config=config.xml --testing, extract the sitemap.xml file from sitemap.xml.gz, and load it in your browser. Look through it and make sure your rules have worked as expected, then run the command again, removing the –testing part. If you want to get fancy, you can set this up as a cron job. If you do, run it on access.log.0, yesterdays logs, around 2am. That way you don’t miss any traffic as the logging switches over at midnight.

    Finally, log into Google webmaster tools and submit your sitemap to Google!

    To see how must of your traffic is coming from the Googlebot, SSH to your server and run tail -10000 access.log| awk '{print $1}' | sort | uniq -c |sort -n from the same directory as your access.log files. The first number is the connections, the second is the IP making those connections. IPs that start with 66.249 are the Googlebot. If 66.249 is the last entry, and the number of connections is very high(over a thousand, say) and many times bigger than the number of connections for the second most frequent IP, you probably need to do something before the hosting company does something for you, like ban Google from accessing your site.
    I’m not exactly sure if it would be better to leave some things in, but set to a zero priority, however I have non-content stuff removed for now. Really, the non-content pages should probably be excluded in robots.txt