Simplifying a Network

June 17th, 2008

I had some time this last weekend to change my home computing platform and for those of us who work independently, I thought I would mention what I ended up doing. First, I’m pretty fortunate in that I have a few neighbors who also work from home. Obviously, our life line is our internet connection to the outside world. When that goes down, it’s hard to work and bill. Last year, we decided to exchange WEP keys with each other so that if one of us loses connectivity, we can easily leach off of someone else’s connection until we are back up. This has worked great for the four of us. Interestingly, out of the four of us, we use three different providers so this has provided even more assurance that we can maintain some connectivity in the event of a connection lose. It’s all about redundancy!

I finally bit the bullet and put to rest my Small Business Server from MS. I’ve used SBS for many years and it has worked OK, but it’s over kill for a one person company. I’ve gotten pretty good at figuring out proxy issues and other communications problems, but I always felt that there had to be something more simple to use. At the encouragement of a colleague, I ordered and installed Windows Home Server from Microsoft. With WHS, I get 10 licenses to connect to the server and I don’t have to worry about the complexity of administration. This thing is pretty simple. Granted, it doesn’t have Exchange or SQL Server but that’s easily overcome. I’m just moving my email to the cloud and since I don’t use SQL Server for anything other than development, I can use the Express Edition to do all my testing and dev work.

I was able to connect my analytics server to WHS and I never lost a beat. So as far as my network environment, I’m pretty much where I want to be without the distraction and overhead of maintaining a network with a domain and Active Directory, etc… One thing that I really like is now that i’m not using Proxy Server, I have full access to other communications methods that was very difficult to implement such as video conferencing.

The other advantage of WHS is that it works seamlessly with Vista and XP for backups and storing media for distribution to other devices. You can also run a website and SharePoint on WHS so you continue to get many of the benefits of SBS for about 16% of the cost WHS.

Printing with WPS

May 30th, 2008

One of my few complaints with WPS is getting LST files/reports formatted properly to print on a printer. I’ve spent quite a bit of time wrestling with WPS not throwing page feeds (or whatever it is called in Windows terminology) so that my printouts fit nicely on a page. If you’ve used WPS, I’m sure you’ve had this issue as well.

Today, I spent a little time playing around with WPS and MS Word 2007. I was originally contemplating writing a short .NET program to read a WPS LST file and format it so that it would fit on a page properly. That is a bit more work than I wanted to do and since I have MS Office 2007 on my desktop, I thought it would be worth a try to set up a macro to format a page properly for printing.

If you don’t have Office 2007, it’s extremely nice.  It is well worth the investment. Although this example uses Word 2007, I’m sure you can do the same thing in Word 2003 that I did using Word 2007. So, how did I finally achieve pages to print properly using WPS? I have an affinity for landscape printing, so I have my options set in my autoexec.sas file as below.

options orientation=landscape ls=132 ps=50 ;
 

This sets up the page margins and orientation for the LST file.

The trick to getting everything to work is to use a FOOTNOTE statement in your code. Again, in my autoexec.sas file I placed the following statement.

FOOTNOTE “Report Provided by MineQuest, LLC.”;
 

Of course, you can use anything you like but I wanted something generic that I could put on any of my printouts in case my programs didn’t contain any footnotes. One gotcha here, your LST file must have a footnote to make this work. If you reset your footnotes to blank, then this little trick will not work.
 

The second part of the equation is setting up MS Word to quickly format the text file that you have to save from within WPS. I created a macro, called WPS_Landscape that simply formats the Word 2007 document so that it is printable. The settings I use are:

·         Orientation: Landscape

·         Top Margin: 0.65″

·         Bottom Margin 0.65″

·         Left Margin 0.5″

·         Right Margin 0.5″

·         Font: Courier New

·         Font Size: 9pt

Setting up the Word Macro

It’s pretty easy to setup a Word 2007 Macro. Here are the steps.

1.       Select the Office Start Button and select Word Options in the lower right hand side of the selection box.

2.       Make sure that you have “Show Developer Tab in the Ribbon” checked.

3.       If the Developer Tab is selected, the Developer options will appear on the Ribbon.

4.       Select the Developer Tab from the ribbon menu.

5.       Click Record Macro.

6.       Type WPS_landscape for the macro name and click the “Keyboard ” icon in the dialog box.

7.       Press F3 (or any key you want to use to launch the macro) for the short cut key that will launch the macro. Press Assign and then Close.

8.       Click Home on the Ribbon.

9.       Click “Select” and then “Select All” on the far right side of the main menu.

10.   Change the font to Courier New and the font size to 9pt.

11.   Click Page Layout and click on Orientation and select Landscape.

12.   Click on Margins and then Custom Margins and enter:

o   Top Margin: 0.65″

o   Bottom Margin 0.65″

o   Left Margin 0.5″

o   Right Margin 0.5″

13.   Click on Developer on the Main Menu and click on Stop Recording.

So there you have it. After you have saved your WPS LST file (as say WPSLST.TXT) all you have to do is right click on the WPSLST.TXT file from Windows Explorer, select “Open with…” and select MS Word. After the text appears in Word, press F3 and your report will be properly formatted to print.

Finally, this also gives you the option of printing to a PDF or RTF from within Word.  You might want to experiment to see how you can add a logo to MS Words header or Footnote bands to make your reports and printouts look quite nice.     

Comparing the Cost of WPS and SAS on a Windows Server

May 19th, 2008

Recently, I decided that perhaps it was time to do a cost analysis of WPS on a Windows Server vs. SAS on the same Windows Server. I looked at the cost for both products for the first year, the annual renewal fee and the total cost of these products over a three year period.

Not surprisingly, the cost of a WPS license was substantially less than SAS for the same products. What was surprising was how much lower the cost for WPS is in comparison to SAS. In a time of budget cutting and when most companies are trying to become lean and mean, SAS is trying to take advantage of its customers by dramatically increasing prices when the customer refreshes hardware. By simply going from a single logical processor to a two logical processor server, SAS increases its pricing by as much as ten times the annual renewal rate.

You can take a look at this White Paper by going to the home page at www.minequest.com. You can directly view the report in PDF format by clicking here.

Tags: WPS SAS Pricing Windows Server

 

More Thoughts on Open Source and Business Intelligence

April 29th, 2008

I’ve been thinking about the last post on Open Source and BI. I’ve had a few discussions with people I respect on this particular subject and I want to share some of the comments and views from those folks as well as how I see this shaking out.

First, most people have taken the perspective that Open Source Software (OSS) means free software and that is the major attraction of Open Source. I too embraced that thought but my views on the value of OSS have been changing. I no longer think the only great value of OSS is that it is free, but that it’s greatest value is that you can decide exactly what you want (for the most part) in your BI stack and how it is to work together. You are no longer locked into the vendor’s view (and at their mercy) that you have to license X if you want Y and Z.

I can’t imagine a better argument for using OSS than to be able to meet your business needs as you see fit. Custom BI solutions is going to be the future for this segment of computing. With OSS, you can have thousands of variations of software that make up your BI stack. Just think about how hard it would be for any vendor to create or produce anything near that for your BI infrastructure.

Of course, using OSS does not mean that you have to ignore or not use proprietary software either. If a vendor is smart, they will embrace OSS so that their clients and customers can extend their BI infrastructure’s functionality. As a matter of opinion, I think that OSS working to extend proprietary software, and in the context of BI solutions, will become dominant as the economy moves towards a more service oriented solution.

Open Source BI

April 15th, 2008

There has been some discussions lately regarding Open Source and whether it has a viable life in BI. I think it does. And it can be a rather large space too. For the last few months, I’ve been kicking around the concept of a JumpBox type system (a VM that contains the applications required for a specific task - see www.jumpbox.com) for BI developers and programmers. I think the concept has some legs. A department could take a BI JumpBox and install it on a server platform and have the rudimentary applications to start doing some real BI work.
 

Consider a JumpBox that has the following components.
 

1.     R for Statistics
2.     MySQL or PostgreSQL for a database
3.     Alfresco or TikiWiki for a developers portal.
4.     Birt or Jasper for Report Writing.
5.     WPS for heavy lifting of data.
 

Each of the above applications would be setup and tuned as part of the JumpBox. One of the reasons that I like WPS so much (and no WPS is not Open Source or free) is that I can extend it quite a bit because of the Eclipse Environment. I’m much more likely able to create some type of plugin that works with Eclipse than I can with SAS’s IDE. If I can create a plug-in then I can most likely integrate with the above apps. Btw, there already is an R plug-in for Eclipse called StatET.
 

I’m really interested in creating one or two vertical market apps that use WPS for the heavy lifting and a suite of applications like I listed above. One is a Demographic Data Center in a Box. The reason that I think makes this viable is the licensing of WPS. If I use WPS for my vertical market app, I as a developer can hold the WPS license if I so choose. I also have the right to transfer that license one time to the client if I wish. This allows me to totally control my application.
 

So to summarize, I do think that Open Source BI probably cannot stand on its own right now. But I think there’s a lot of room for integration of Open Source with some Closed Source apps to enhance theirvalue to the client. Perhaps large companies would “poo-pah” the idea of using Open Source in this manner, but I’m confident that smaller organizations would not consider it to be such a problem.

The Value of a Consultant

March 26th, 2008

I often go onsite and in the course of my work I often work with other independent SAS Consultants. I find that one of the virtues of working with good SAS consultants and one of the things that we have in common is that we possess a vast library of code in the form of macros. I have approximately 300 macros in my library that I’ve created and collected over the years. Some of them I’ve posted on my website but I keep most of them proprietary. On a recent consulting contract, one of the other SAS consultants told me that he has almost 600 macros which he had stored on a USB drive.

The value of a good SAS consultant is that they are able to bring to the table existing code that is proven honed and polished.  Code reuse saves the client money. And let me tell you, it can be a considerable sum of money when you don’t have to reinvent the wheel at each site.

Which brings me to the point of today’s blog posting… a couple new macros.  When putting SAS or WPS Systems in to production, you often find yourself looking for some functions that are common in other languages but don’t exist in the current development environment. For example, getting the filename from a string, the file extension for the string, or the pathname from the string.

I’ve written three short macros that perform these functions in SAS and/or WPS.  The macros are:

Filename - Returns the filename from a string.

PathName - Returns the pathname from a string.

FileExt - Returns the file name extension from a given string.

The actual code for the macros are below.

%Macro filename(instr);
    reverse(scan(reverse(&instr),1,’\'))
%mend filename;

%Macro PathFileName(instr);
      substr(&instr,1,length(&instr)-length(%filename(&instr))-1)
%mend PathFileName;
 

%Macro FileExt(instr);
    reverse(scan(reverse(%Filename(&instr)),1,’.'))
%mend FileExt;
  

 And here is a bit of SAS code that demonstrates how to use each of the above macros.

  

data _null_;
length fn path1 path2 $ 100;
PathFileName = ‘C:\subdir1\subdir2\subdirX\MyfileName.txt’;
 fn = %fileName(PathFileName);
 path1 = %PathFileName(pathfilename);
 Path2 = %pathFileName2(pathfileName);
 Fext = %fileExt(PathFileName);
 put ‘File Name = ‘ fn ;
 put ‘Path FileName = ‘ Path1;
 put ‘File Extension=’ Fext;
run;
 
 
 

 

 

 

The Analytics Pro Package

March 8th, 2008

Recently, I came across the Analytics Pro Package from SAS Institute. This is a package that the Institute offers to small and medium sized businesses (SMB tier) at discounted prices. This package is usually offered through a few of their resellers who are authorized to sell to companies that have annual revenues at $500 million or less.

Let’s take a look at the “Analytics Pro Package” which is licensed for $6,700 and contrast it with what you can have from World Programming Company complimented with the Open Source Statistical package R.

I’m sure many of you are sick of reading about ROI…. But it truly is instructive to take a look at pricing comparisons to understand value.

SAS Product
WPS Product
Comments
SAS/Base WPS/Base

WPS contains almost all of what SAS has in their Base product.

SAS/STAT R

We substitute R for SAS/STAT and use the free WPS2R Bridge.

        

SAS/Graph

        

WPS and R

WPS has basic support for PROC GPLOT and GCHART. Otherwise, we will use R for graphics.

        

SAS/Access (limited to two access products)

15 library engines to access other data formats including ODBC and SAS data sets.

        

What can I say?

     

Single User Price: $6,700

$660

Savings:  $6,040

Five User Price: $33,500

$3,300

Savings: $30,200

By using the World Programming System (WPS) in lieu of SAS, you have the advantage of saving thousands of dollars and potentially 10’s of thousands of dollars if you have as few as five users. You have the advantage of continuing to use the SAS language that you have grown so accustomed to using but with the advantage of a superior IDE.

 

 

 

WPS2R Installation Guide now Available

March 8th, 2008

I finally got the chance to update the WPS2R Bridge Installation Guide. The guide now reflects the changes to how the Bridge works as well as change so that the Bridge works in interactive mode. You can find the guide in the downloads section of the MineQuest web site  at:  www.MineQuest.com. The guide can be found at: http://www.minequest.com/downloads/Installation_WPS2R_Bridge.pdf.

Also, if you have any questions on how to install the WPS2R Bridge, feel free to send an email to the support address listed in the back of the guide. I’ll be happy to try to help you get the Bridge functional. I will state upfront however that I can’t help you on your R coding problems. I’m not an R expert by any means (pun intended!)

I’ve received about a half dozen emails on the Bridge already and they’ve all been centered on two basic questions. I thought that I would share them with you. The questions are:

Q. Are you going to make the Bridge work with SAS?

A. Not at for a while. If you want to consider running R inside your IDE, please consider licensing a copy of WPS.

Q. Will you make the Bridge work with binary data sets where I don’t have to export the data set each time to a CSV file?

A. That’s on the agenda and will be under development soon. It’s definitely possible to do and I don’t see any technological issues preventing this from happening.

WPS2R Bridge Now Available

March 3rd, 2008

I finished up the WPS2R Bridge and have posted the Bridge on the MineQuest website. The WPS2R Bridge allows WPS users to write R code in line with WPS code and execute the R code, with the output and listing file from R being routed back to the WPS output and listing windows. You can see a short three minute demo of this facility at:

 http://www.minequest.com/downloads/WPS2R_Bridge.html

Being pressed for time, I put together a three page quick guide to installing and using the bridge. I’ll write a complete document but I’m under pressure to get a system rewritten and converted in the next few weeks so jobs that pay get the priority!

The WPS2R Bridge fills a gap that I think is important to address. You can now perform some high level statistical analysis using WPS and R. For years, SAS users have relied on SAS/STAT, ETS, and other libraries to perform their analysis. With R, you can perform many types of statistical procedures that have yet to make it into the SAS libraries. SAS has pretty much left their stat libraries sit for years and have placed most of their emphasis on developing products like Enterprise Miner. Not many people or organizations can afford to pay annual fees that SAS charges for these products.

R has libraries for time series analysis, linear and non-linear modeling, clustering, Bayesian statistics among others and are available for download from the R website or mirrors. By using R, you can have access to the most cutting edge statistical techniques now, and not when some software company decides it’s time to code it for you.

Other reasons that I chose to develop the WPS2R Bridge is that R is rapidly gaining in popularity. R is not a fad and is now being used outside of academic circles and finding its way into the corporate world. It was recently estimated that 30% of the new stat jobs being posted are asking for some exposure and familiarity with R. Younger statisticians are often the ones who desire to use R as opposed to SAS. I believe this may be a harbinger of things to come.

Finally, using WPS with R is fairly natural. You can use WPS for data management and reporting and by using the WPS2R Bridge, you now have access to high level statistical procedures and graphics.

You can download the WPS2R Bridge in a zip file format at http://www.minequest.com/downloads.html

Let me know your thoughts on the package and how it may be improved.

Links:
World Programming Company

WPS2R Bridge Documentation

WPS2R Bridge Zip File

R Statistical Package

More on a WPS to R Bridge

February 28th, 2008

At the beginning of the month, I wrote a posting about creating a bridge so that WPS users can easily and transparently run R Statistics from within the WPS Workbench. The code that I wrote and published was pretty much a proof of concept that this could be done. However, it was painfully obvious to anyone who wanted to use it that it was a clumsy solution. It worked but it was awkward!

One of the drawbacks to the first version of the bridge was that it didn’t allow you to take advantage of R’s graphics abilities in an interactive fashion. It also forced the programmer to write R code and include the filename (R code is placed in a separate text file) in the header of the macro. I don’t know about most of you, but when I’m analyzing data, I want to focus on the problem at hand and not have to deal with jumping through hoops in my programming. I’m pretty much an interactive and iterative kind of developer.

I’m quite a bit further ahead in the development of a new process where you can run R code directly from within the WPS environment. As a matter of fact, I would estimate that I’m 90% of the way there. I need to clean up the code and I want to add one more option so that there’s some flexibility in how the data is handed off to R. I also need to write some documentation for it.

Most of the code will be available when released with the exception of a small executable that I was forced to write to handle some housekeeping chores. The executable does a number of functions such as check for the existence of files and directories, rename files, create directories and call the R program without the dos command box flashing around on your screen. I’ll document the executable and the macros that make use of it so that you can include it in your own library of functions. I think you will find many of the functions valuable if you are writing or converting systems to run in a production environment.

So, if you wish to see how the WPS2R Bridge works, you can view a short three minute video on the MineQuest web site at: http://www.minequest.com/downloads/WPS2R_Bridge.html.