Yiping's profileYiping's spaceBlog Tools Help

Blog


    October 20

    Use your computer smartly

    We always feel our laptop can open a file it in seconds. Yet as the FYP goes on, we suddenly find sometimes the file is so large or the calculation is so complex that waiting become necessary. But trust me, our small laptop can still handle most of the cases. It’s just that you have to choice the right tool.

    Last night, a friend ask me to make a C++ program because her Perl script has been running forever. The task is easily understood: inner product of two tables.

    image

    The bottleneck is that one of the table contains 40,000 records and she was using a n^2 complexity method (search the whole table for each match). I don’t want to look at the Perl script or code a C++ during such a relaxing night because the first idea came into my mind is SQL. Yes, it is nature to do this in SQL.

    I host Microsoft SQL 2008 Express on my old T43 so I just import the two tables then do the simple query:

    SELECT * FROM [Table1], [Table2]

    WHERE [ID1] = [ID2]

    No doubt the database engine was highly optimized for doing such things and the whole process was finished in 6 min.

    Comments (5)

    Please wait...
    Sorry, the comment you entered is too long. Please shorten it.
    You didn't enter anything. Please try again.
    Sorry, we can't add your comment right now. Please try again later.
    To add a comment, you need permission from your parent. Ask for permission
    Your parent has turned off comments.
    Sorry, we can't delete your comment right now. Please try again later.
    You've exceeded the maximum number of comments that can be left in one day. Please try again in 24 hours.
    Your account has had the ability to leave comments disabled because our systems indicate that you may be spamming other users. If you believe that your account has been disabled in error please contact Windows Live support.
    Complete the security check below to finish leaving your comment.
    The characters you type in the security check must match the characters in the picture or audio.

    To add a comment, sign in with your Windows Live ID (if you use Hotmail, Messenger, or Xbox LIVE, you have a Windows Live ID). Sign in


    Don't have a Windows Live ID? Sign up

    wrote:
    actually for this case, an array is most efficient than hash...
    Oct. 24
    wrote:
    In fact, I just realized I did the same thing in my FYP using perl on more than 60000 lines of molecular descriptors. A simple hash structure will do......Peal is unbelievably elegant sometimes! haha~ I'll die without it...
    Oct. 24
    Erchun Wangwrote:
    but, i don't understand, why you can achieve this without using inner join? SQL server will defaul the linkage as inner join if you use where to specify the search criterion
    Oct. 20
    Erchun Wangwrote:
    i did the same thing last week, will explain to you a bit later in email
    Oct. 20
    Qi Meiwrote:
    Brilliant
    Oct. 20

    Trackbacks

    The trackback URL for this entry is:
    http://kookaburracc.spaces.live.com/blog/cns!76D91D7030633430!2611.trak
    Weblogs that reference this entry
    • None