AWS S3 - Example of searching files in S3 using regex and the ResponseStream

There have been times when I've needed to inspect contents of text files that were created as map reduce output and stored in S3. I had been downloading the files, but there were hundreds of files and they were all very big (around 360 MB each). It was a hassle since it would take a long time to download every file, and it wasted a lot of diskspace. I wanted a way to search for certain data, and then cancel my search so I could stop downloading so much data.  
The solution I chose to use was to use a Regex against the ResponseStream available when you do a GetObject call. That way I'm downloading data, but it isn't being stored on my computer.

Here is the main bit of code for searching the objects contents:

private void SearchObjectForString(AmazonS3 amazonS3, string bucketName, string key, string searchString)
{
    Cursor.Current = Cursors.WaitCursor;

    // Issue call
    var request = new GetObjectRequest();
    request.BucketName = bucketName;
    request.Key = key;

    using (var response = amazonS3.GetObject(request))
    {
        using (var reader = new StreamReader(response.ResponseStream))
        {
            string line;
            var rgx = new Regex(searchString, RegexOptions.IgnoreCase);

            while ((line = reader.ReadLine()) != null)
            {
                Application.DoEvents();
                if (cancelled)
                {
                    Cursor.Current = Cursors.Default;
                    return;
                }

                var matches = rgx.Matches(line);
                if (matches.Count > 0)
                {
                    lstResults.Items.Add(string.Format("{0}/{1}:{2}", request.BucketName, request.Key, line));
                }
            }
        }
    }

    Cursor.Current = Cursors.Default;
}

Regex in C# to get UTC date...

I recently needed to find a UTC date in a string of text, and I thought it might be handy to pull the various values from the dates that were found by getting the values from the returned Groups for each Match. Groups are identified in your regular expression string by surrounding sections with parentheses. The first Group of every match is the string value that the Match found. Every other group in the Match is identified by the parentheses going from left to right. So, if you have a regular expression that looks like this:

"((\d\d\d\d)-(\d\d)-(\d\d))T((\d\d):(\d\d):(\d\d))Z"

Imagine that someone used the above regular expression on the following string.

"This is a UTC date : 1999-12-31T23:59:59Z. Get ready to party!"

The very first group (after the matched date/time string) will be the entire date, because the first set of parentheses completely wraps the date portion of the string.

"1999-12-31"

The next group would be the year portion of the date, since the next set of parentheses completely wraps the year.

"1999"

That pattern is repeated for the rest of the regular expression string. If no parentheses (groupings) are specified, then there will only be the one group and it will contain the string that the regular expression matched. Here is an example of how to do this in code:

static void Main(string[] args)
{
    string input = "this\tis\ta test 2013-03-21T12:34:56Z\tand\tanother date\t2013-03-21T23:45:01Z";
    string regexString = @"((\d\d\d\d)-(\d\d)-(\d\d))T((\d\d):(\d\d):(\d\d))Z";
    TestRegex(input, regexString);
}

private static void TestRegex(string input, string regexString)
{
    int matchCount = 0;
    foreach (Match match in Regex.Matches(input, regexString))
    {                
        int groupCount = 0;
        foreach (Group group in match.Groups)
        {
            Console.WriteLine("Match {0}, Group {1} : {2}", 
                                matchCount, 
                                groupCount++, 
                                group.Value);    
        }
        matchCount++;
    }
}

Here is the output:

Match 0, Group 0 : 2013-03-21T12:34:56Z
Match 0, Group 1 : 2013-03-21
Match 0, Group 2 : 2013
Match 0, Group 3 : 03
Match 0, Group 4 : 21
Match 0, Group 5 : 12:34:56
Match 0, Group 6 : 12
Match 0, Group 7 : 34
Match 0, Group 8 : 56
Match 1, Group 0 : 2013-03-21T23:45:01Z
Match 1, Group 1 : 2013-03-21
Match 1, Group 2 : 2013
Match 1, Group 3 : 03
Match 1, Group 4 : 21
Match 1, Group 5 : 23:45:01
Match 1, Group 6 : 23
Match 1, Group 7 : 45
Match 1, Group 8 : 01