Silent Archimedes

Posts Tagged ‘Ubuntu’

Problem: Maximum number of files in NTFS in Ubuntu

Posted by silentarchimedes on March 12, 2009

So I do a lot of image processing in my work, and I ran into a problem today that I still haven’t found a logical answer to:

On my Ubuntu 8.10 computer, I have a 1TB internal harddrive /dev/sdc1 mounted in NTFS format. After the initial format, My max volume size is stated as 931.51GiB. On this drive, I have lots of folders, and many of them have thousands (1K to 6K) of small images, in formats of jpg, png, ppm. Today, one of my scripts crapped out when it tried to create a new image and returned “Operation not supported.”

Even when I used touch or a simple vi created file, I could not create any more files. The current disk usage on the harddrive is:

Contents: 704,324 items, totalling 826.3 GB

And Ubuntu tells me I still have 102.5 GiB of space left. So I started thinking if I’ve reached my inodes limit because of the number of files. However, when I do a ‘df -i‘, I am no where near the limit:

Filesystem           1K-blocks      Used Available Use% Mounted on
/dev/sdc1            108159004  706236 107452768    1% /media/sdc1

When I look online, all the documentation and search says that the maximum number of files in NTFS is 2^32-1. The only other thing is the master file table (MFT) and how it might increase it’s size if more files get created beyond those specified in MFT. However, I haven’t been able to confirm this.

Anybody have an explanation for this? It’s really bugging me that it’s happening and I can’t figure it out.

Posted in Computers, Linux, Technology | Tagged: , , , , , , , , , , , | 3 Comments »

DBP, mogrify, convert – How to batch crop, resize, rename, format convert images in Ubuntu

Posted by silentarchimedes on March 11, 2009

DBP – David’s Batch Processor for GIMP

If you want to do this in the context of GIMP, download and install DBP (David’s Batch Processor). It will show up as an option under the Filters menu list. Just click ‘Batch Process’ and a GUI will pop up. You simple add files to the input list and you can do various basic image processing operations on it. They include any combinations of rotate, blur, colorize, resize, crop, sharpen, rename and image format conversion.

One drawback to DBP is that it does not allow you to add a directory or directories instead of a list of individual images. For some people that need to batch process directories of images, you will have to either manually do a directory one at a time or temporarily put all your images into one directory. This is a bit of a pain.

mogrify or convert – ImageMagick tools for Linux

If you are more of a command line guy or if you do need to batch process directories of images, mogrify or convert is the way to go. The man page of mogrify states, ‘mogrify – resize an image, blur, crop, despeckle, dither, draw on, flip, join, re-sample, and much more. Mogrify overwrites the original image file, whereas, convert(1) writes to  a  different image file.’

You can simply put a bunch of mogrify commands into a script file and let it run in the background. An example mogrify command to resize all your jpegs to 256×256 looks like:

  mogrify -resize 256x256 *.jpg

An example convert command to resize all your jpegs to 256x256 gifs with a prefix of images looks like:

  convert -size 256x256 *.jpg images%0d.gif

Look  at the ImageMagick’s mogrify page or convert page for more info. Speaking of, if you don’t have ImageMagick installed on your Ubuntu system, you should. 🙂

What if you want to command line convert images and put them in another directory, but keep the same names as the original images?

So at first it seemed like convert was the way to go since mogrify is supposedly only for modifying the original images in place. However, convert‘s way of doing it requires a bit of linux scripting. There is a easier in mogrify. It  has an option -path that allows you to specify an output image path.

In the following command, I want to crop out a 320×480 subimage beginning at location (160,0) in all the ppms in the tempim directory. I want the processed images to have the same names as the originals but to put them in the tempim2 directory:

mogrify -path tempim2 -format png -size 640×480 -extract 320×480+160+0 tempim/*.ppm

Posted in Computers, Linux, Technology | Tagged: , , , , , , , , , , , , , , , | 7 Comments »

How I got Vista working in Ubuntu 7.10

Posted by silentarchimedes on May 9, 2008

What info are you interested in?

1. I want to install Vista on Ubuntu
2. I want to install Ubuntu on Vista
3. I am looking for a Vista img or VMWare software
4. I am getting Vista img or ACPI errors
5. I am getting VMWare errors
6. I am getting KVM or QEMU errors
7. I am looking for Soundblaster drivers
8. Other

View Results

Please vote so I can add information that people are looking for!

Background:

I bought a Dell Optiplex 745 early 2007 loaded with Vista Ultimate for my research. However, since I am more of a Linux person, I dual booted my computer with Ubuntu 6.10 Edgy Eft. I didn’t upgrade to 7.04 because I wanted to keep using Apache 1.x for my web server (pre-forking support). When I took down my webserver end of last year, I finally updated my Ubuntu to 7.04 and then 7.10. I have not updated to Ubuntu 8.04 Hardy Heron yet.

In any case, I realized I hardly boot up in Vista because it requires a physical reboot and I usually have a bazillion windows and workspaces up in Linux. As I started missing my Windows games and apps, I thought about purchasing another computer, strictly for Windows. But that was too pricey. I had used wine (a Windows emulator on Linux), but that seemed unreliable at times and doesn’t support all Win apps. Then I started investigating about virtual machines, and this article will talk about how I got Vista working in Ubuntu 7.10 using QEMU and then VmWare.

What is a virtual machine?

In short, a virtual machine (VM) is a software implementation of a machine (computer) that executes programs like a real machine. For example, I can run Vista or XP as a VM inside Linux. It also allows me to test out other Linux distros without creating a whole partition for it, since VMs run as a single file in an existing partition. VMs are faster and run more natively then emulators. See the virtual machine wiki page for more info.

My Computer:

Dell Optiplex 745 Core2 Duo 2.13Ghz, 2Gb RAM, 2x250Gb WD HDD, ATI Radeon x1300, Windows 32-bit Vista Ultimate, Ubuntu 7.10, Dell UltraSharp 2407 LCD, Linux Kernel 2.6.22.14

What I Needed:

Microsoft Vista installation DVD
VMWare Workstation or VMWare Player+VMTools
QEMU (I tried this first)
CreativeLabs Soundblaster PCI (ES1371, ES1373) Windows driver

How I got Vista working in Ubuntu 7.10:

This is not a step-by-step path that you should follow if you want to get Vista working on Ubuntu. This is the step-by-step I went through to get it working. You can decide if you need to do them or not. For example, you can go straight to using VMWare software instead of trying open-source software like QEMU. And I’m not sure how much upgrading my BIOS had to do with the process. Maybe it’ll help you if you get stuck.

There are also other ways to getting Windows to work in Linux. Websites such as EasyVMX create virtual machines online for you. However, I have never tried that and don’t know how well they work.

1. Upgraded my BIOS

What led me to upgrading my BIOS was in regards to whether my computer supported virtualization and ACPI (Advanced Configuration and Power Interface) or not. Most tutorials first ask you to turn it on in your BIOS. As I went into my BIOS, the only option I had was to turn on Intel’s Virtualization Technology option. Before going any further, I decided to upgrade my Dell to the latest BIOS. I suggest doing this as it can’t really hurt. I did later find out that my BIOS and computer naturally supports virtualization and there was no need to turn anything on in my BIOS.

Another way to check if your computer supports virtualization, the following command in Linux should be a non-empty result:

grep -E ‘^flags.*(vmx|svm)’ /proc/cpuinfo

Links:
Emulating Vista under QEMU
How Windows Determines ACPI Compatibility
The BIOS in this system is not fully ACPI compliant

2. Install QEMU/KQEMU

Even though I ended up using VMWare software, I still used QEmu (with kqemu as the acceleration driver) along the way to do useful things (such as creating an .img and installing a Vista .iso into it). I also tried just using Qemu for my virtualization. So, go ahead and install QEMU.

  1. Follow steps 1-9 of the Ubuntu Tutorial – Running WIndows XP under QEMU.
  2. I also installed qemu, kvm,kqemu-source, kqemu-modules-2.6.22-15-generic and any other dependencies they have.

Links:
QEMU
Installing QEmu, KQEMU and patches
QEMU (KVM) and Vista (very useful)
Running WIndows XP under QEMU (very useful)

3. Create a vista.img

In QEMU, the VM will reside in a .img file “drive”. I first created a .img file in my ~username/vm directory. Then I installed a Vista OS into the .img file. The one main option in creating an “empty” .img file is the format, raw or qcow. In raw, you specify a set size that is not expandable. However, you can mount the VM in Linux using ntfs-3g. In qcow, you specify an initial size (Vista requires atleast 6.8Gb minimum), and the VM will expand in 2Gb increments when necessary. However, it is not mountable.

If you are going to use VMWare, I would suggest using the fat format. We will convert the .img file into .vmdk format later anyways, so don’t worry about size now. If you are going to use just QEMU, then you can try the qcow format, but when I tried that, I had a lot more of the errors below than when I used the fat format.

In any case, you create a .img using the following command. What is bold can be changed by you. Then change the permissions to be writable.

qemu-img create -f fat vista.img 9G
chmod 660 vista.img

4. Install Vista into the vista.img file

The next step is to install Vista into the newly created img file. First thing is to have your Vista DVD ready. I used the DELL reinstallation DVD that came with my computer. There are two options here, installing directing from the DVD into the Vista.img file or first create a Vista.iso file. The latter option is faster, since it doesn’t require continuous DVD access. To create an iso file, use the following command:

dd if=/dev/cdrom of=vista.iso

If your cdrom device is not cdrom, just change accordingly. You can check by seeing where it is mounted, or check the /dev directory. My cdrom is mounted in /media/cdrom. Remember, in Linux, cdrom device refers to both cd-rom and dvd-jrom drives.

Now to install the Vista.iso into the Vista.img file, use one of the following commands. I’ve had varying luck with all of them. First a new window opens up that has a progress bar and “Windows loading files” text. Then Vista attempts to load up for installation. If you skipped creating the iso file and want to install directly from dvd, replace vista.iso with /dev/cdrom.

qemu-system-x86_64 -boot d -m 512 vista.img -cdrom vista.iso

  • The qemu-system-x86_64 command can be used for 32-bit machines also since kvm does not differentiate between the 32-bit and 64 bit machines.
  • The -m specifies how much memory to use.
  • I ran into the following errors and was never able to overcome the second error:
  1. A write to read-only memory blue error screen. This is because the img file does not have write permissions. Use chmod 755 vista.img to allow the current user to write to it.
  2. A PAGE_FAULT_IN_NONPAGED_AREA blue error screen. An old forum had some ideas regarding this error.

kvm -m 1024 -localtime -net nic,model=rtl8139 -net user,vlan=1 -cdrom vista.iso -boot d vista.img

  • -localtime forces QEMU to use the local time of the host Ubuntu computer.
  • -net nic,model=rtl8139 sets the networking to the Network Interface Controller (nic) interface, with model of Realtek PCI Fast Ethernet rtl8139 driver
  • If you get the following error in your xterm when you run the command, ignore it for now:
    open /dev/kvm: No such file or directory
    Could not initialize KVM, will disable KVM support
  • The troubling thing with this method is I kept getting the PAGE_FAULT_IN_NONPAGED_AREA blue error screen over and over again. If you keep trying, you might get lucky. For some reason, after like 10 tries, it gets past it and the Vista Install Window shows up (see below). I still don’t know why and when it does it. One interesting possibility might be because Vista does not produce valid boot sector information. This person installs XP on the img first, before installing Vista over it. You can try that but since I didn’t have an XP iso available I couldn’t verify it. If you do try and it works, please let me know.
  • Go ahead and install Vista now.

qemu -m 384 -localtime -cdrom vista.iso -boot d vista.img

  • This command actually had the most luck for me, at least getting into the Vista installation process. However, I still would sometimes get the PAGE FAULT blue error screen when Vista has to reboot during installation. If anyone knows why this happens, please let me know.

5. Run Vista in Ubuntu using QEMU

If you’ve gotten this far, you are almost there. You can actually run Vista in Ubuntu right now. The following command would boot up Vista in barebones QEMU-style:

qemu -localtime -m 512 vista.img

  • If you use the following options, you will need to install the drivers for them first.
  • -soundhw es1371 emulates the Creative Labs ES1371 sound card.
  • -net nic,model=rtl8139 treats your ethernet card as a Realtek PCI Fast Ethernet rtl8139 driver
  • -usb -usbdevice tablet treats your mouse like a tablet, so instead of clicking on the Vista window to enact it, you can simply move the mouse into the window.

6. Installing VMWare Player/Workstation

QEMU has some shortcomings that I couldn’t resolve. One was getting the sound to work (although I didn’t try that hard in installing the sound driver). The other was that it only has a VGA graphics adapter, which forces the Vista window to be at max 1280×1024. This was a big deal to me since my LCD native is 1920×1200.

So I moved on to try using VMWare’s free virtualization software, the VMWare Player (and eventually VMWare Workstation). It’s actually a nifty little GUI and tool, but what makes it annoying is that it requires you to load a virtual machine that is created in other VMWare software (all of which are not free). However, this is more of a technalicality, since all it requires is a VM in .vmx and .vmdk formats. We can convert our .img VM to these formats using QEMU’s utilities.

Download VMWare Player from VMWare’s website.

7. Convert vista.img to vista.vmdk

After installing the Player, we have to convert the .img file to vmdk format, which is VMWare’s hard disk format.

qemu-img convert vista.img -O vmdk vista.vmdk

8. Create a Vista.vmx – the VMWare vm configuration file

Finally, we create a vmx file that configures the virtual machine vmdk. We can specifies settings for sound cards, memory, etc in here. However, you can leave the vmx file barebones and use the VMWare GUI to add to the file. Use this link to create a barebones vmx file.

Links:

How to create virtual machines using VMWare Player

9. Success and Beyond

Once you’ve created the vmx file, it is pretty straightforward at this time. Just start VMWare Player and open the virtual machine specified by your vmx config file. There is plenty of info online on how to customize your virtual machine. Some things you might run into is getting your sound to work. Try rebooting sometimes. I remember nothing working and a simple reboot gave me sound for good. I also had to install the Soundblaster PCI 128 (ES1371) driver directly in Vista. Since that driver is way out of date, you won’t find the link directly from the Create Labs website. You’ll have to search the web for it. I’ll put a link up here when I get around to it.

Additionally, the main reason you want to possibly move up to VMWare Workstation is the VMTools it comes with. The VMTools contains the SVGA graphics adapter driver that allows higher res VMs. Some people have gotten around this by downloading the free trial for the Workstation, using the VMTools, and then downgrading back to Player.

Finally, I list some websites below of related information that might makes things even easier for you.

Create virtual machines online – http://www.easyvmx.com/
VMWare Player
HOWTO – VMWare-Server 1.04 and Kernel 2.6.23.1 on Gutsy…
VMWare How to

Please respond to the poll below. Also, let me know if you want more information added to the post. I am always updating old posts. Thanks!

How helpful was this post
1) not at all
2) a little
3) fairly
4) pretty helpful
5) very helpful

View Results

Posted in Technology | Tagged: , , , , | 16 Comments »