Friday, March 14, 2014

Building PoDoFo for Qt using MinGW

It is better to use the shipped MinGW in Qt, if you are going to use it in a Qt project. Open a command line by either typing mingw on your windows search or locate the installed Qt folder; then run inside an opened command line the following file: bin/qtenv2.bat. This will nicely setup your environment to use the MinGW shipped with Qt.

Better yet, use MSYS (http://www.mingw.org/wiki/Getting_Started).

Install the cool synaptic-like tool (mingw-get-setup.exe) to install MSYS and friends.
Then use it to install MSYS. No need to install MinGW because we will use the ones shipped with Qt.
Lastly, point the location of MinGW to the one that Qt ships with. This is important to avoid potential headaches. Create fstab inside /etc/:
#Win32_Path Mount_Point
d:/ProgramData/Qt/Tools/mingw48_32 /mingw

Dependencies

zlib (http://www.zlib.net/)

Qt already has zlib so no need to build for this. Yey!
Or, if you want to use the latest:
Download the windows compiled dll zip.
As stated in USAGE.txt, you can run the following to create a static library:
dlltool -D zlib1.dll -d lib/zlib.def -l lib/libzdll.a 

libjpeg (http://www.ijg.org/)

Using msys, just do the standard compilation jutsu:
./configure --prefix=/mingw
make
make install

libtiff (http://www.remotesensing.org/libtiff/build.html)

Using msys, just do the standard compilation jutsu:
./configure --prefix=/mingw --with-jpeg-include-dir=/mingw/include --with-jpeg-lib-dir=/mingw/lib
make
make check
make install

freetype (http://freetype.sourceforge.net/index2.html)

Using msys, just do the standard compilation jutsu:
./configure --prefix=/mingw --host=i686-pc-mingw32
make
make install

Ignore the rmdir errors..

PoDoFo

openssl 

Use the tar in msys so that the symbolic references will be preserved.
Using msys, just do the standard compilation jutsu:
./config
make
make test

libpng (http://libpng.sourceforge.net/index.html)

Copy makefile.msys from scripts/ to the package root directory with name Makefile.
Then just run make.

Cmake (http://www.cmake.org/files/v2.8/)

Because of a bug in the latest cmake (or the build script of PoDoFo is outdated..), we must use an older version: 2.8.3. http://www.daniweb.com/software-development/cpp/threads/438435/windres.exe-invalid-option-w-

Finally, to build PoDoFo itself

Create a podofo build dir:

mkdir podofo-debug

Run cmake for mingw:
podofo-debug>cmake.exe -G "MinGW Makefiles" ..\podofo-src -DCMAKE_INCLUDE_PATH=<Qt mingw install dir>\include;<Qt mingw install dir>\i686-w64-mingw32\include -DCMAKE_LIBRARY_PATH=<Qt mingw install dir>\lib;<Qt mingw install dir>\i686-w64-mingw32\lib -DPODOFO_BUILD_SHARED:BOOL=FALSE

Run make.
Extract headers and libraries built.

ToDo:
For release, libjpg-9.dll is searched by windows, I don't know the reason yet..
libjpg-9.dll is located in .libs/ of the libjpg build.

Tuesday, January 15, 2013

Setup w3af on Ubuntu 12.04 LTS

Update (2013/06/11)

It seems that I missed the dev package for python, I added it below.
Also, if pip fails for some reason and the apt commands are available, you can try your luck with apt-get.
Thanks to the good folks who commented below!

Install pip to install dependencies

sudo apt-get install python-pip

Install dependencies

sudo apt-get install python2.7 python2.7-dev
sudo pip install fpconst
sudo pip install nltk
sudo pip install SOAPpy
sudo pip install pyPdf
sudo apt-get install libxml2-dev
sudo apt-get install libxslt-dev
sudo pip install lxml
sudo pip install pyopenssl
(Download from http://www.secdev.org/projects/scapy/)
sudo pip install scapy-latest.tar.gz
(Download from http://pysvn.tigris.org/)
sudo pip install pysvn-1.7.6.tar.gz
or
sudo apt-get install python-svn, if on Ubuntu
sudo pip install pybloomfiltermmap

GUI Dependencies

sudo apt-get install graphviz
sudo apt-get install libgraphviz-dev
sudo apt-get install libgraphviz
sudo apt-get install python-gtk2
sudo apt-get install python-gtksourceview2

Finally, w3af

svn co https://w3af.svn.sourceforge.net/svnroot/w3af/trunk w3af
./w3af_console
or
./w3af_gui

Enjoy!


References

[1] http://survivalbit.com/blog/article/w3af-virtualenv-osx/

Friday, April 27, 2012

How to implement a Duplex Service between ASP.NET and WCF

Un/fortunately, I am working on a Microsoft technology based project for a year now. While the .NET environment looks good on the surface, if you start doing something cool, then you have to either pay for their support or lean on the shoulders of community MVPs. Because the project has no budget for premium Microsoft support, I had to lean a lot on community MVPs and Google mostly (I'm not a Microsoft fan, as you can see).

 I was trying to do something that should be fairly easy, that is implementing a Duplex Service back end for an ASP.NET front end. In this setup, the back end service does some heavy computational processing that reports updates regarding its status to the ASP.NET front end. A normal basicHttpBinding can also be used, but I wanted the back end service to be scale-able. The strategy is that possibly multiple front-end services can send requests to possibly multiple back end services and will only report updates on the status of the request to front-end services that sent them. Also, I'm too lazy to manually configure each back-end service for every front-end service that exists. Now that I have explained the main drive for this, I should be getting back to the topic at hand..

MSDN [1] is actually a good place to start for finding resources on how to quickly create working prototypes. Basically, we just need to create a ServiceContract that has a callback interface and required session mode.

    [Serializable]
    public class TaskResponse
    {
        public string TaskID;
        public bool Successful;
        public string Message;
    }

    /// Contract for Task Service
    [ServiceContract(Namespace = "Example.Service.Duplex/", SessionMode = SessionMode.Required,CallbackContract = typeof(ITaskServiceCallback))]
    public interface ITaskService
    {
        /// Order slave to do pushups
        [OperationContract]
        TaskResponse DoPushUps(int number);
    }

    /// Callback for Task Service contract
    public interface ITaskServiceCallback
    {
        /// Report the result of the task
        [OperationContract(IsOneWay = true)]
        void TaskComplete(TaskResponse result);
    }

On the server side, we need to set the InstanceContextMode to PerSession and use the OperationContext of the current session to use the callback.

    /// Provides services for a task master
    [ServiceBehavior(InstanceContextMode = InstanceContextMode.PerSession)]
    public class TaskService : ITaskService
    {
        private static int timerBase = 1000;
        private static Object randSync = new Object();
        private static Random rand = new Random();

        public TaskService()
        {
        }

        /// Quickly get the operation context of the request that sent this request
        ITaskServiceCallback Callback
        {
            get
            {
                return OperationContext.Current.GetCallbackChannel();
            }
        }

        #region ITaskService Members
        public TaskResponse DoPushUps(int number)
        {
            // Create push up task
            string id = Guid.NewGuid().ToString();
            var pushUpTask = new PushUps(id, Callback)
            {
                NumberOfPushUps = number
            };
            // Do task in another thread
            Thread t = new Thread(new ThreadStart(() => { TaskSlave(pushUpTask); }));
            t.Start();
            return new TaskResponse()
            {
                TaskID = pushUpTask.ID,
                Successful = true,
                Message = ""
            };
        }
        #endregion

        /// Task slave to order around!
        private void TaskSlave(Task t)
        {
            // do push ups!
            var pushUp = t as PushUps;
            if (pushUp != null)
            {
                int duration;
                lock (randSync)
                {
                    duration = (timerBase * rand.Next(3)) * pushUp.NumberOfPushUps;
                }
                Thread.Sleep(duration); // simulate long processing task
                pushUp.Callback.TaskComplete(new TaskResponse() { 
                    TaskID = pushUp.ID,
                    Successful = true,
                    Message = "I am tired master"
                });
                return;
            }
            // slave doesn't know the task
            pushUp.Callback.TaskComplete(new TaskResponse()
            {
                TaskID = pushUp.ID,
                Successful = false,
                Message = "I could not understand your order master"
            });
        }
    }

That's it for the server side! For the client side, we need to create a realization for the callback. It is important to note here that for this to work on ASP.NET, we must add a callback behavior property[2].


        [CallbackBehavior(UseSynchronizationContext = false)]
        class TaskServiceCallbackHandler : ITaskServiceCallback
        {
            public void TaskComplete(TaskResponse result)
            {
                Console.WriteLine("Task Complete for " + result.TaskID+ " with result: " + result.Successful);
            }
        }

Then to call the server, you can just extend from ClientBase but you must also specify an InstanceContext.


        class TaskServiceClient : ClientBase, ITaskService
        {
            public TaskServiceClient(InstanceContext instanceContext, string endPointName, string address)
                : base(instanceContext, endPointName, address)
            {

            }

            public TaskResponse DoPushUps(int number)
            {
                return base.Channel.DoPushUps(number);
            }

        }

        InstanceContext ic = new InstanceContext(new TaskServiceCallbackHandler());
        Target = new TaskServiceClient(ic, "Example.Service.TaskService.Dual", "http://localhost:50190/DuplexService.svc");

For the configuration file, you must use a binding that supports sessions like the wsDualHttpBinding and specify the clientBaseAddress. The following is an example configuration:



Please take note of the timeouts. The receiveTimeout of wsDualHttpBinding is important. By default it is set to 10 minutes. If there are no communication within the timeout, the session will expire and the client will throw a ApplicationException. [3] In order to deal with this, the client must reconnect again with a new InstanceContext.

Also, you might encounter exceptions telling that the application is not authorized bind to a URI or something similar. To handle this, just run the following in a shell with an Administrator privilege:

netsh http add urlacl url=http://+:[port number]/[service name] user=[user]

Just replace the text in between [] with a correct one.

I have created a sample project with a unit test of the service. You can download it here.

References
[1] http://msdn.microsoft.com/en-us/library/ms731184.aspx
[2] Calling a Duplex Contract From ASP.NET
[3] http://blogs.msdn.com/b/madhuponduru/archive/2007/02/24/how-to-change-wcf-session-time-out-value.aspx

Wednesday, March 2, 2011

Zend Self Contained Modules

So I've been playing around with Zend lately; and since my last web project was back in college, I was surprised how much development has happened! Where was I when all of this happened? Back then there were no Joomla nor Drupal, only "hand-woven" CMS! Anyways, I'm here back to square one and trying to relearn everything; luckily/unluckily, I picked Zend as my framework of choice.

Compared to Yii, I was surprised that modules in Zend are very hard to implement. What I wanted was a way to initialize my resources using a module specific configuration file; and a way to run code specific for a module (i.e. add actions to an action stack). Thanks to guidelines from Matthew[1] and module configurators from Padraic[2]: I implemented my own module configurator.

Here is my directory structure:
./
application/
 configs/
  application.ini
 layouts/
modules/
 default/
  configs/
   module.ini
  controllers/
  forms/
  layouts/
  library/
 Configurator/
  Layout.php
  models/
  views/
  helpers/
  scripts/
  Bootstrap.php
 news/
  Bootstrap.php
  docs/
  library/
MyApp/
 Controller/
 Plugin/
  ModuleConfigurator.php
 public/
 tests/
Similar from [2], I registered a plugin at Bootstrap.php:
class Bootstrap extends Zend_Application_Bootstrap_Bootstrap
{
 protected function _initControllerPlugins()
 {
  // Register module configurator so that it will run first and initialize the modules
  $frontController = Zend_Controller_Front::getInstance();
  $frontController->registerPlugin(new MyApp_Controller_Plugin_ModuleConfigurator(),1);
 }
}
The Module Configurator does the following:
class MyApp_Controller_Plugin_ModuleConfigurator extends Zend_Controller_Plugin_Abstract
{
  protected $_view;
  protected function moduleInit($_bootstrap, $_config)
  {
   // Run resource configurators

   $resources = array_keys($_config->resources->toArray());
   foreach ($resources as $resourceName)
   {
    $options = $_config->resources->$resourceName;
    $configuratorClass = $this->_view->originalModule . '_Library_Configurator_' . ucfirst($resourceName);
    $configurator = new $configuratorClass($options);
    $configurator->setBootstrap($_bootstrap);
    $configurator->init();
   }

  // Attach plugins
  $plugins = $_config->plugins->toArray();
  foreach ($plugins as $pluginName)
  {
   $frontController = Zend_Controller_Front::getInstance();
   $frontController->registerPlugin(new $pluginName());
  }
 }

 public function dispatchLoopStartup(Zend_Controller_Request_Abstract $request)
 {
  // Get view
  $viewRenderer = Zend_Controller_Action_HelperBroker::getStaticHelper('viewRenderer');
  $viewRenderer->init();
  $view = $viewRenderer->view;
  $this->_view = $view;
  // set up common variables for the view
  $view->originalModule = $request->getModuleName();
  $view->originalController = $request->getControllerName();
  $view->originalAction = $request->getActionName();
  $front = Zend_Controller_Front::getInstance();
  $bootstrap = $front->getParam('bootstrap');
  $moduleName = $request->getModuleName();
  $moduleDirectory = Zend_Controller_Front::getInstance()->getModuleDirectory($moduleName);
  $configPath = $moduleDirectory . DIRECTORY_SEPARATOR . 'configs' . DIRECTORY_SEPARATOR . 'module.ini';
  if (file_exists($configPath))
  {
   if (!is_readable($configPath))
   {
    throw new Exception('modules.ini is not readable for module "' . $moduleName . '"');
   }
   $config = new Zend_Config_Ini($configPath, $bootstrap->getEnvironment());
   $this->moduleInit($bootstrap, $config);
  }
 }
}
So basically, what it does is that it reads the module.ini of the module, then init() every resource. Also, attach plugins if given in the module.ini. So a resource initiliazor is just a realization of a Zend_Application_Resource_ResourceAbstract, like below:
class News_Library_Configurator_Layout extends Zend_Application_Resource_ResourceAbstract
{
 public function init()
 {
  $layout = $this->getBootstrap()->getResource('layout');
  $layout->setOptions($this->getOptions());
 } 
}
Here is an example of a module.ini:
[production]
resources.layout.layout = "layout"
resources.layout.layoutPath = APPLICATION_PATH "/modules/Core/layouts/scripts"
plugins[] = "MyApp_Controller_Plugin_ViewSetup"
Each module has a specific library by autoloading it from a module bootstrap, like below:
class News_Bootstrap extends Zend_Application_Module_Bootstrap
{
 protected function _initLibraryAutoloader()
 {
  return $this->getResourceLoader()->addResourceType('library','library','Library');
 }
}
If all of these seems advanced to you, check out the references below, these guys explained these stuffs in great detail and I don't want to steal their spotlight (actually I'm just too lazy!)

 If I miss something, just tell me! :)

 References:
 [1] http://www.weierophinney.net/matthew/archives/234-Module-Bootstraps-in-Zend-Framework-Dos-and-Donts.html
[2] http://blog.astrumfutura.com/2009/09/self-contained-reusable-zend-framework-modules-with-standardised-configurators/
[3] http://framework.zend.com/wiki/pages/viewpage.action?pageId=16023853
[4] http://blog.vandenbos.org/2009/07/07/zend-framework-module-config/
[5] http://binarykitten.me.uk/dev/zend-framework/177-active-module-based-config-with-zend-framework.html

Wednesday, January 19, 2011

Setting up default routes and nameservers on Ubuntu 10.x

Usually I set up dns entries using the network connection gui but somehow I found myself needing to manually set it up. Also, there are certain routes that I want to be added by default on the kernel routing table. Now it's been ages since I tried to do this and it surprised me how a lot of things have changed. Learning is really a continuous and painful but fun process :)

First I needed to setup my default routes; after googling and reading man pages [1], I found out that you can add scripts that can be ran during bringing up/down of an interface. So what I did was I added a post-up command on the interface that needed static routes. Now the scripts that are run are stored at /etc/network/[event], where [event] can be if-down.d, if-post-down.d, if-up.d, if-post-up.d. I named my script routes.sh; below is a sample interface entry:

# The primary network interface
auto eth1
iface eth1 inet static
address x.x.x.x
netmask x.x.x.x
network x.x.x.x
broadcast x.x.x.x
gateway x.x.x.x
# dns-* options are implemented by the resolvconf package, if installed
dns-nameservers x.x.x.x x.x.x.x
dns-search artesyncp.com emrsn.org
# Static routes
post-up /etc/network/if-up.d/routes.sh

routes.sh:
#!/bin/bash

ip route flush dev eth1
ip route add to x.x.x.x/24 dev eth1
ip route add to x.x.x.x/16 via x.x.x.x dev eth1
ip route add to x.x.x.x/16 via x.x.x.x dev eth1

Of course x.x.x.x should be replaced by a valid IP address.

The problem was there is also an interface that retrieves its address through a DHCP server but the DNS entries that it sets are not complete. I can't touch the DHCP server and /etc/resolv.conf always gets overwritten during network reboots. Because of this, I needed to add option modifiers to my /etc/dhcp3/dhclient.conf. Below are what i added:

prepend domain-search "something.com", "another.something.com";
prepend domain-name-servers x.x.x.x, x.x.x.x;

I used prepend because I want to use my static name servers before the one that the DHCP server gives.

References

[1] man 5 interfaces
[2] man 5 dhclient.conf
[3] man 5 dhcp-options

Monday, March 15, 2010

Bump into Shift Operation Oddity

Recently, we bumped into an odd behavior on a shift operation. A code we were debugging was doing a left shift on an integer (1) for 125 times. We were expecting the result to be 0 but oddly the result was 536870912. After consulting our friend Google, we got this explanation:

C99 standard, section 6.5.7 "Bitwise shift operators",
page 84:
"If the value of the right operand is negative or is
greater than or equal to the width of the promoted left
operand, the behavior is undefined."

Apparently, the operation was undefined and the behavior is implementation specific. We tried running the code in C and Java; and found out that if the operator is greater than the operand, in this case the size of the operand is 32, the operand is shifted by "operator modulo size-of-operand" times (1 % (125 mod 32)).

I guess we learn new things everyday! ^_^

Thursday, January 14, 2010

Code a division operation by a specific value without a division operator?

Even though the semester is already done, I've decided to continue this blog because its a great way to keep track of what I've learned and practice my writing. Also, because I didn't start the year right, this is my way of getting myself back on track!

Anyway, the subject of this post was asked to me and my colleagues by one of our seniors. He told us that back then there weren't much and they encountered this odd problem while on a project. At first, subtracting the dividend by the divisor while counting the number of times it has been subtracted seemed like a good idea but this solution is very slow. We got the right solution thanks to my colleague IC.

Objective: Divide any given number by a specific number without using the divider operation.

Conditions:
1. The process of multiplication, addition and bit shifting are available.
2. The memory registers are larger than the size of the integer to be divided.
3. Floating point numbers are not supported.

Sample Case: Divide any given 16-bit integer by 48 using a 32-bit register.

Solution:

Let x be the given number, d be the divisor and c be the constant multiplier which is divisible by two.

Hint: x/d is just equal to (x*c)/(d*c )

1. Determine a constant value of 1/48 which will be used as part of the multiplier. Note that a number divided by 48 is equal to that same number multiplied by 1/48:

1/48 = 0.020833333333333333333333333333333

2. Choose a multiplier that will not cause an overflow by dividing the size of the register to the maximum integer value. We are just looking for a specific multiplier that when multiplied by 0xFFFF (the maximum value for a 16 bit integer), will not be greater than 0xFFFFFFFF (the 32-bit register):

(0xFFFF * 1/48)c <= 0xFFFFFFFF so,
c = 0xFFFFFFFF / (0xFFFF * 1/48) = 0x300030 = 0b1100000000000000110000

3. Reduce this number into the highest number divisible by two because we can only use shift operations:

0b1100000000000000110000 can be reduced to 0b1000000000000000000000 which can easily be obtained since this is just 1 << 21.

4. Multiply the inverse constant of the divider by the number above:

0.20833333333333333333333333333333 * 0b1000000000000000000000 = 43690

5. To prevent any round down errors, round up the constant multiplier by adding one:

43690 + 1 = 43691

6. Multiply this constant to the given integer number and divide it with the same multiplier from step 3.

The final equation would then be the following:

(x * 43691) >> 21
Which is accurate until 1 << 15

I'm not a Math wizard so corrections, comments and suggestions are very much welcomed! ^_^