RamDisk driver: make_request, concurrency, udelay, alloc_page?

Timbuktu85 · 05-31-2012, 08:17 AM

Hi all,

I'm really new in Linux, especially in driver development. Now I'm trying to write some sort of ramdisk driver (similar to sbull (LDD3), brd and RapidDisk). This should however be a simulator for a real block device.

Following questions are for me now really problems.

----------------------------1--------------------
I use a "no queue" mode, i.e., directly implement make_request function. The simulated device supports concurrent IOs and have certain time latencies for READ/WRITE ops. (which should be simulated). The concurrent behaviour is based on the device structure, e.g. the device consists of two separate parts, which could process the IO separatly (and in parallel) (within a certain "part" requests are syncronous/sequential). For the delay simulation I've used udelay function (because usleep_range is very inaccurate for my purposes, i.e., 25us, 100us ...). Now the question: could the atomic udelay be used in such concurrent contex. Here is the code I'm thinking about (simplified

):

Code:

void make_request(struct request_queue *q, struct bio *bio)
{
    getnstimeofday(start);     
    spinlock_t lock = <<find a lock for a certain "device part" based on bio>>;
    spin_lock(lock);
    //simulate IO ....   
    getnstimeofday(end);
    udelay(needed - (end - start));
    spin_unlock(lock);
}

Without udelay this seems to be (to my mind

) a make_request function that supports parrallel requests between device "parts"(locks for each device part).

Would udelay broke this statement? I mean: could udelay be executed in PARALLEL for two (3, 4, ...) IO requests? Would it be atomic in "current_lock" context? Or it will destroy the concurrency behaviour? (Sorry for such a dummy questions - I'm a new one in Linux).
How many parallel udelays could be executed? (#CPU ?)

----------------------------2--------------------
When is the transferred data available for user_space (i.e., for application that issues IO):
- after I've copied the data to buffer (e.g. memcpy);
- or after the call to bio_endio(bio, status) ???
this has impact on where I should put delay function.

----------------------------3--------------------
As I've understand allocating a driver memory (in my case - device memory) with vmalloc is not a good idea (bcz of mapping overhead), especially if I need huge amound of data (up to 50 Gb). Would alloc_page (indexed with RADIX_TREE) work fine for 50Gb (of course under the condition of availability of such amount of RAM)? Are there other variants?

Thanks a lot for your help?

Best,
Tim

sundialsvcs · 05-31-2012, 04:48 PM

You most certainly cannot "delay" while holding a spinlock, because every other CPU that wanted that lock would be dead in the water.

Basically, what you have here is a requirement for what I call "forced-delay scheduling." (Not a linux term. Don't go looking for it...) In other words, every request must carry with it a time-of-day value, such that the request must not be carried out until at least that time-of-day has passed. If you want to impose, say, a simulated "100 msec delay," you must somehow stamp the request with an indicator that this request must not be honored until at least "100 msec from right-now." Then, you must arrange some mechanism by which the queue of pending requests will be re-considered "not later than" this point in time.

One way to approach the problem is to arrange for one timer-driven event to occur at a time-of-day that roughly coincides with the time-of-day of the earliest pending request. Each time this event happens, it scans the entire queue of pending requests, and either executes or marks-for-execution all those which are ready to happen. (Then, if it has determined that any requests are yet-to-occur, it must reschedule itself to occur at the appropriate time-of-day.)

Timbuktu85 · 06-01-2012, 02:38 AM

Thanks for your answer. I think I've get your idea.

You said:

Quote:

Originally Posted by sundialsvcs

You most certainly cannot "delay" while holding a spinlock, because every other CPU that wanted that lock would be dead in the water.

- but is this true for different locks? Suppose my device consists of two separate, individual parts, which could process requests in parallel. So I will maintain two separate locks: lock1 and lock2. Once I receive a bio (in make_request function), I first figure out to which part of device this request belongs and based on this acquire either a lock1 or lock2. E.g., if lock1 is hold while the bio for the second part arrives - the process could acquire the lock2 and process further with the delay. Is it so?

Could I achieve a parallelism based on acquiring different locks? Would the udelay works in such scenario in parallel?

P.S. the delays I need to simulate are from 20 to 200 microseconds (for 4K pages), and as well as it should be a simulator I need to try to do this with the best precision.

Could your also help me with other two questions?

Thanks.