Blog title image disappeared...

I have no idea what happened, but my blog title image disappeared.  I'm not sure how long it was missing, but perhaps a couple of days. I ended up uploading the image to a website, and then using the URL to reference where the image was located.  

I found some recent information from others where their images disappeared, but nothing that seemed identical to my situation. The other mentions of missing images all pointed to Picasa web albums being deleted, and that was why the images were missing. I didn't delete anything recently, so I don't think that was my issue. However, I assumed that when the layout control gives you an option to select an image from your computer that it was copying the image to whatever disk space is used to host your blog entries.  Perhaps that was a bad assumption.

Has anyone else experienced the problem of blogger title images disappearing?

Using Amazon's AWS S3 via the AWS .Net SDK

Amazon's AWS S3 (Simple Storage Service) is incredibly easy to use via the AWS .Net SDK, but depending on your usage of S3 you might have to pay. S3 has a free usage tier option, but the amount of space allowed for use is pretty small by today's standards (5GB). The upside is that even if you end up going outside of the parameters for the free usage tier it is still cheap to use.

Here is some information from Amazon regarding the free usage tier limits for S3:

• 5 GB of Amazon S3 standard storage, 20,000 Get Requests, and 2,000 Put Requests
• These free tiers are only available to existing AWS customers who have signed-up for Free Tier after October 20, 2010 and new AWS customers, and are available for 12 months following your AWS sign-up date. When your free usage expires or if your application use exceeds the free usage tiers, you simply pay standard, pay-as-you-go service rates (see each service page for full pricing details). Restrictions apply; see offer terms for more details.

Sign Up To Use AWS

You need to create an account in order to use the Amazon Web Services. Make sure you read the pricing for any service you use so you don't end up with surprise charges. In any case, go to http://aws.amazon.com/ to sign up for an account if you haven't done so already.

Install or Reference the AWS .Net SDK

To start using the AWS .Net SDK to access S3 you will want to either download the SDK from Amazon or use NuGet via Visual Studio. Start Visual Studio (this example is using Visual Studio 2010), and do the following to use NuGet to fetch the AWS SDK:

• Select the menu item "Tools | Library Package Manager | Manage NuGet Packages For Solution..."
• Type "AWS" in the "Search Online" search text box
• Select "AWS SDK for .Net" and click the "Install" button
• Click "OK" on the "Select Projects" dialog

Create a Project and Use the AWS S3 API

Create a project in Visual Studio, and add the following code:

string key = "theawskeythatyougetwhenyousignuptousetheapis";
string secretKey = "thesecretkeyyougetwhenyousignuptousetheapis";

// create an instance of the S3 TransferUtility using the API key, and the secret key
var tu = new TransferUtility(key, secretKey);

// try listing any buckets you might have
var response = tu.S3Client.ListBuckets();

foreach(var bucket in response.Buckets)
{
   Console.WriteLine("{0} - {1}", bucket.BucketName, bucket.CreationDate);

   // list any objects that might be in the buckets
   var objResponse = tu.S3Client.ListObjects(
      new ListObjectsRequest 
      {
         BucketName = response.Buckets[0].BucketName
      }
   );

   foreach (var s3obj in objResponse.S3Objects)
   {
      Console.WriteLine("\t{0} - {1} - {2} - {3}", s3obj.ETag, s3obj.Key, s3obj.Size, s3obj.StorageClass);
   }
}

// create a new bucket
string bucketName = Guid.NewGuid().ToString();
var bucketResponse = tu.S3Client.PutBucket(new PutBucketRequest
   {
      BucketName = bucketName
   }
);

// add something to the new bucket
tu.S3Client.PutObject(new PutObjectRequest
   {
      BucketName = bucketName,
      AutoCloseStream = true,
      Key = "codecog.png",
      FilePath = "C:\\Temp\\codecog.png"
   }
);

// now list what is in the new bucket (which should only have the one item)
var bucketObjResponse = tu.S3Client.ListObjects(
   new ListObjectsRequest
   {
      BucketName = bucketName
   }
);

foreach (var s3obj in bucketObjResponse.S3Objects)
{
   Console.WriteLine("{0} - {1} - {2} - {3}", s3obj.ETag, s3obj.Key, s3obj.Size, s3obj.StorageClass);
}

Micro ORM Review - FluentData

Who doesn't love tools that make your life easier? Make a database connection and populate an object in just a few lines of code and one config setting? That's what FluentData can offer. Sign me up! 

Some Features:

• Supports a wide variety of RDBMs - MS SQL Server, MS SQL Azure, Oracle, MySQL, SQLite, etc.
• Auto map, or use custom mappers, for your POCOs (or dynamic type).
• Use SQL, or SQL builders, to insert, update, or delete data.
• Supports stored procedures.
• Uses indexed or named parameters.
• Supports paging.
• Available as assembly (download the DLL or use NuGET) and as a single source code file.
• Supports transactions, multiple resultsets, custom return collections, etc.

Pros:

• Setting up connection strings in a config file, and then passing the key value to a DbContext to establish a connection is such an easy way to do things.  It made it very easy to have generic code point to various databases.  I'm sure that is the intent. Needing to declare a connection object, set the connection string value for the connection object, and then calling the connection object's "Open" method seems undignified. :D It's really not that big of a deal, but I like that it seemed much more straight forward using FluentData.
• It's very easy to start using FluentData to select, add, update, or delete data from your database.
• It is easy to use stored procedures.
• Populating objects from selects, or creating objects and using them to insert new data into your database is almost seamless.
• Populating more complex objects from selects is fairly easy using custom mapper methods.
• The exceptions that are thrown by FluentData are actually helpful. The contributors to/creators of FluentData have been very thoughtful in how they return error information.

Cons:

• I had some slight difficulty setting a parameter for a SQL select when the parameter was used as part of a "like" for a varchar column.  The string value in the SQL looked like this: '@DbName%'.  I worked around the issue by changing the code to use this instead: '@DbName', and then set the value so that it included the %.

I originally thought that I couldn't automap when the resultsets return columns that don't map to properties of the objects (or are missing columns for properties in the target object) without using a custom mapping method. However, there is a way - you can call a method on the DB context to say that automapping failures should be ignored:

Important configurations

• IgnoreIfAutoMapFails - Calling this prevents automapper from throwing an exception if a column cannot be mapped to a corresponding property due to a name mismatch.

Example Usage:

First, I created a MySQL database to use as a test. I created a database called ormtest, and then created a couple of tables for holding book information:

create table if not exists `authors` (
  `authorid` int not null auto_increment,
  `firstname` varchar(100) not null,
  `middlename` varchar(100),
  `lastname` varchar(100) not null,
  primary key (`authorid` asc)
);

create table if not exists `books` (
 `bookid` int not null auto_increment,
 `title` varchar(200) not null,
 `authorid` int,
 `isbn` varchar(30),
 primary key (`bookid` asc)
);

Next, I created a Visual Studio console app, added an application configuration file, and added a connection string for my database:

Then I created my entity types:

public class Author
{
 public int AuthorID { get; set; }
 public string FirstName { get; set; }
 public string MiddleName { get; set; }
 public string LastName { get; set; }

 public string ToString()
 {
  if (string.IsNullOrEmpty(MiddleName))
  {
   return string.Format("{0} - {1} {2}", AuthorID, FirstName, LastName);
  }
  return string.Format("{0} - {1} {2} {3}", AuthorID, FirstName, MiddleName, LastName);
 }
}

public class Book
{
 public int BookID { get; set; }
 public string Title { get; set; }
 public string ISBN { get; set; }
 public Author Author { get; set; }

 public string ToString()
 {
  if (Author != null)
  {
   return string.Format("{0} - {1} \n\t({2} - {3})", BookID, Title, ISBN, Author.ToString());
  }
  return string.Format("{0} - {1} \n\t({2})", BookID, Title, ISBN);
 }
}

I was then able to populate a list of books by selecting rows from the books table:

public static void PrintBooks()
{
 IDbContext dbcontext = new DbContext().ConnectionStringName("mysql-inventory", new MySqlProvider());
 const string sql = @"select b.bookid, b.title, b.isbn
         from books as b;";
   
 List<Book> books = dbcontext.Sql(sql).QueryMany<Book>();

 Console.WriteLine("Books");
 Console.WriteLine("------------------");
 foreach (Book book in books)
 {
  Console.WriteLine(book.ToString());
 }
}

Unfortunately I wasn't able to select columns from the table that didn't have matching attributes in the entity type. You'll need to create a custom mapping method in order to select extra columns that don't map to any attributes in the entity type. You can also use custom mapping methods to populate entity types that contain attributes of other entity types).

Here is an example:

public static void PrintBooksWithAuthors()
{
 IDbContext dbcontext = new DbContext().ConnectionStringName("mysql-inventory", new MySqlProvider());

 const string sql = @"select b.bookid, b.title, b.isbn, b.authorid, a.firstname, a.middlename, a.lastname, a.authorid 
         from authors as a 
        inner join books as b 
        on b.authorid = a.authorid 
        order by b.title asc, a.lastname asc;";

 var books = new List<Book>();
 dbcontext.Sql(sql).QueryComplexMany<Book>(books, MapComplexBook);

 Console.WriteLine("Books with Authors");
 Console.WriteLine("------------------");
 foreach (Book book in books)
 {
  Console.WriteLine(book.ToString());
 }
}

private static void MapComplexBook(IList<Book> books, IDataReader reader)
{
 var book = new Book
 {
  BookID = reader.GetInt32("BookID"),
  Title = reader.GetString("Title"),
  ISBN = reader.GetString("ISBN"),
  Author = new Author
  {
   AuthorID = reader.GetInt32("AuthorID"),
   FirstName = reader.GetString("FirstName"),
   MiddleName = reader.GetString("MiddleName"),
   LastName = reader.GetString("LastName")
  }
 };
 books.Add(book);
}

And here is an example of an insert, update, and delete:

public static void InsertBook(string title, string ISBN)
{
 IDbContext dbcontext = new DbContext().ConnectionStringName("mysql-inventory", new MySqlProvider());

 Book book = new Book
 {
  Title = title,
  ISBN = ISBN
 };

 book.BookID = dbcontext.Insert("books")
         .Column("Title", book.Title)
         .Column("ISBN", book.ISBN)
         .ExecuteReturnLastId<int>();

 Console.WriteLine("Book ID : {0}", book.BookID);
 
}

public static void UpdateBook(Book book)
{
 IDbContext dbcontext = new DbContext().ConnectionStringName("mysql-inventory", new MySqlProvider());
 book.Title = string.Format("new - {0}", book.Title);

 int rowsAffected = dbcontext.Update("books")
        .Column("Title", book.Title)
        .Where("BookId", book.BookID)
        .Execute();

 Console.WriteLine("{0} rows updated.", rowsAffected);
}

public static void DeleteBook(Book book)
{
 IDbContext dbcontext = new DbContext().ConnectionStringName("mysql-inventory", new MySqlProvider());

 int rowsAffected = dbcontext.Delete("books")
        .Where("BookId", book.BookID)
        .Execute();

 Console.WriteLine("{0} rows deleted.", rowsAffected);
}

Summary:
FluentData has been fairly easy to use and there appears to be a way to accomplish whatever I want to do. If FluentData's documentation had more examples of how to populate entity types (POCOs), then it would have saved me a little bit of time. As it is, the documentation listed multiple ways to accomplish tasks, so it never took long to find a method that would work.

sed to the rescue...

I had one of those dreaded tasks at work today - update a bazillion (okay, actually just 24, but it felt like a lot) servers with a config file that needs to be modified to reference the name of each of the servers. I either had to update each server in turn, or I could write a script to do it for me.  

I decided to write a script, and I will pretend that it took less time to write and run the script than doing it manually.  It wasn't perfect - I had to copy/paste the password for each server because I didn't want to mess around with using keygen. I also didn't want to install software like expect.  Expect is a software utility that allows you to write scripts that provide interactive programs with the parameters they might...well...expect.  In this case it would have been handy to use since it could provide scp with the user's password. The script I wrote did what I needed it to, and I'm sure I'll be able to use something like this script in the future.    

Assuming that the source file is on server01, and the source file to be updated and copied is named somefile.conf, then this is what the script looked like:

#!/bin/sh

servers="server02 server03 server04 server05 server06 server07 server08 server09 server10 server11 server12 server13 server14 server15 server16 server17 server18 server19 server20 server21 server22 server23 server24"
filepath="/some/file/path/somefile.conf"

for server in $servers
do
      echo "Working on $server."
      SEDARG="s/server01/$server/g"
      sed $SEDARG $filepath > $filepath.$server
      scp $filepath.$server someuser@$server:$filepath
      rm $filepath.$server
done
echo "Done."

Solr - HTMLStripCharFilter...

I am attempting to store a bit of data that I fetch from a website in Solr.  The data sometimes has HTML markup, so I decided to use the HTMLStripCharFilterFactory in the fields analyzer.

Here is an example of the field type that I created:

<fieldType name="strippedHtml" class="solr.TextField">
   <analyzer>
      <charFilter class="solr.HTMLStripCharFilterFactory"/>
      <tokenizer class="solr.WhitespaceTokenizerFactory"/>      
      <filter class="solr.LowerCaseFilterFactory" />
   </analyzer>
</fieldType>


I used the field type of strippedHtml in a field called itemDescription, and when I do a search after indexing some data I can see that the itemDescription contains data that still has HTML markup.  I used the analyzer tab in Solr to see what would happen on index of HTML data, and I could see that none of the markup appears to be stripped out.

It turns out that most of the HTML was encoded so that the angle bars are replaced with the escaped values.  I will need to find a way to remove the escaped values.

Code puzzler on DZone and optimizations...

Something that I've really enjoyed about DZone.com is that they have recurring themes for certain posts. One recurring set of posts is the Thursday Code Puzzler. Now and then there will be a really interesting solution. For example, one puzzler was to count the number of 1's that occurred in a set of integers. ie, {1, 2, 11, 14} = 4 since the number 1 occurs 4 times in that set. One of the solutions used map reduce to come up with the solution. I thought that was particularly neat.

I decided to try out the recent code puzzler for finding the largest palindrome in a string. Here was my first method:

public static int getLargestPalindrome(String palindromes) {

        String[] palindromeArray = palindromes.split(" ");
        int largestPalindrome = -1;

        for(String potentialPalindrome : palindromeArray) {
            if (potentialPalindrome.equals(new StringBuffer(potentialPalindrome).reverse().toString()) && potentialPalindrome.length() > largestPalindrome)
                largestPalindrome = potentialPalindrome.length();
        }

        return largestPalindrome;

    }

It works, but it felt like cheating to use the StringBuffer.reverse() method. Here is my second method:

    public static int getLargestPalindrome(String palindromes) {

        int largestPalindrome = -1;

        for(String potentialPalindrome : palindromes.split(" ")) {
            int start = 0;
            int end = potentialPalindrome.length() - 1;
            boolean isPalindrome = true;

            while (start <= end) {
                if (potentialPalindrome.charAt(start++) != potentialPalindrome.charAt(end--)) {
                    isPalindrome = false;
                    break;
                }
            }

            if (isPalindrome && potentialPalindrome.length() > largestPalindrome) {
                largestPalindrome = potentialPalindrome.length();
            }

        }

        return largestPalindrome;

    }

It works faster than the first method. I'm sure the performance improvement is mainly due to the first method's creation of new StringBuffer objects (and also new Strings for the reversed value) for each potential palindrome, but accessing locations in an array for half the length (worst case in 2nd version) is bound to be less work than (worst case in 1st version) comparing every character in a string to another string.

Thread pool example using Java and ExecutorService...

Using thread pools is something that is very easy to implement using Java's ExecutorService. The Java ExecutorService class allows you to specify the number of asynchronous tasks that you want to process. Here is an example of an ExecutorService class being instantiated where numThreads is an integer specifying the number of threads to create for the thread pool:

ExecutorService executorService = Executors.newFixedThreadPool(numThreads);

You pass a runnable in the ExecutorService's execute method like this:

executorService.execute(someRunnableObject);

I created a sample method that uses an ExecutorService to similute working on text files. It will move files from the source path to an archive path unless there is a "lock" file found. The "lock" file is an empty file that is named identically to one of the files that is being "worked" on. The lock file is used to ensure that the same file isn't attempted to be worked on by multiple threads. I made this sample because I figured this might be a nice way to handle indexing data in csv files to a Solr server. Here is the method that does the work (which would be very poorly named if it weren't sample code):

public static void DoWorkOnFiles(String sourcePath, String archivePath, int numThreads) throws IOException {

    Random random = new Random();
    ExecutorService executorService = Executors.newFixedThreadPool(numThreads);

    File sourceFilePath = new File(sourcePath);
    if (sourceFilePath.exists()) {
        Collection<java.io.File> sourceFiles = FileUtils.listFiles(sourceFilePath, new String[]{"txt"}, false);

        for (File sourceFile : sourceFiles) {
            File lockFile = new File(sourceFile.getPath() + ".lock");
            if (!lockFile.exists()) {
                executorService.execute(new SampleFileWorker(sourceFile.getPath(), archivePath, random.nextInt(10000)));
            }
        }
        // This will make the executor accept no new threads
        // and finish all existing threads in the queue
        try {
            executorService.shutdown();
            executorService.awaitTermination(10000, TimeUnit.MILLISECONDS);
        } catch (InterruptedException ignored) {
        }
        System.out.printf("%nFinished all threads.%n");
    }
    else {
        System.out.printf("%s doesn't exist. No work to do.%n", sourceFilePath);
    }
}

The SampleFileWorker class looks like this:

import org.apache.commons.io.FileUtils;
import java.io.File;
import java.io.IOException;

public class SampleFileWorker implements Runnable {

    private final String sourcePath;
    private final String archivePath;
    private final int testDelay;

    public SampleFileWorker(String sourcePath, String archivePath, int testDelay) {
        this.sourcePath = sourcePath;
        this.archivePath = archivePath;
        this.testDelay = testDelay;
    }

    @Override
    public void run() {

        try {
            File lockFile = new File(sourcePath + ".lock");
            if (!lockFile.exists()) {
                lockFile.createNewFile();
            } else {
                return;
            }

            File sourceFile = new File(sourcePath);
            String archiveFilePath = archivePath.concat(File.separator + sourceFile.getName());
            File archiveFile = new File(archiveFilePath);

            System.out.printf("Simulating work on file %s.%n", sourcePath);
            System.out.printf("Starting: %s%n", sourcePath);
            System.out.printf("Delay:    %s%n", testDelay);

            try {
                Thread.sleep(testDelay);
            } catch (InterruptedException ignored) {
            }

            System.out.printf("Done with: %s%n", sourcePath);
            System.out.printf("Archiving %s to %s.%n", sourceFile, archivePath);

            FileUtils.moveFile(sourceFile, archiveFile);
            sourceFile.delete();
            lockFile.delete();

        } catch (IOException e) {
            e.printStackTrace();
        }
    }
}